Method and system to create custom, user-specific eyewear

ABSTRACT

Systems and methods for creating fully custom products from scratch without exclusive use of off-the-shelf or pre-specified components. A system for creating custom products includes an image capture device for capturing image data and/or measurement data of a user. A computer is communicatively coupled with the image capture device and configured to construct an anatomic model of the user based on the captured image data and/or measurement data. The computer provides a configurable product model and enables preview and automatic or user-guided customization of the product model. A display is communicatively coupled with the computer and displays the custom product model superimposed on the anatomic model or image data of the user. The computer is further configured to provide the customized product model to a manufacturer for manufacturing eyewear for the user in accordance with the customized product model. The manufacturing system is configured to interpret the product model and prepare instructions and control equipment for the manufacturing of the customized product.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application Nos.61/869,051, filed Aug. 22, 2013, and 62/002,738, filed May 23, 2014, theentire contents of which are hereby incorporated by reference.

FIELD OF THE INVENTION

The invention pertains to the on-demand creating, manufacturing, anddelivering one-up custom products from scratch. More particularly, thesubject invention creates, manufactures, and delivers custom personalproducts on-demand that are best suited to the needs and preferences ofan individual user by building the product from a specification that isgenerated from automatic and/or user-guided user-specific preferenceprofiles and by building a unique one-up custom product based on theprofiles.

BACKGROUND OF THE INVENTION

Although there are many personal products that one might want to havecustomized or made as a one-of-a kind product tailored to a particularuser, a key one of these personal products is eyewear. While theinvention will be described in connection with creating producing anddelivering custom eyewear, it will be appreciated that the subjectinvention involves the creation, production and delivering of a widevariety of products that relate to the anatomical or physicalcharacteristics of the user as well as the users preferences forparticular product. That having been said, it will be appreciated thatdescribing the invention in terms of the creation, production anddelivery of eyewear carries a large number of similarities to thecreation, production and delivering of a wide variety of productscustomized to the features and desires of the user. What followstherefore describes invention in terms of eyewear, it being understoodthat the invention is not so limited.

Purchasing eyewear, while a necessity for many people, presents manychallenges for consumers. For traditional in-store purchases, consumersare faced with limited in-store selection, which often requires visitingmultiple stores. Yet users must explore an unmanageable array of optionsto find a compromise between fit, style, color, shape, price, etc.Eyewear is most commonly mass-produced, with a particular styleavailable in one or two generic colors and sizes. Users faces are uniqueenough that a face can be used as a primary form of identification, yetthey must choose between products made for a generic faces that are nottheir own. It is very difficult for users to find the one perfect pairof glasses for their unique taste, facial anatomy, and needs. They alsooften have difficulty visualizing what they try on because they need anoptical prescription in the first place.

Recent entrants have explored the online marketplace for eyewear in anattempt to address some of these issues. However, none of thecommercially available eyewear selection systems attempt to provide acompletely unique one-up, from-scratch product that is customized to theuser's anatomical features, as well as the user's likes and dislikes.There is therefore a need to provide a user with a completelycustomizable one up product that does not rely on only off-the-shelfpreviously designed mass-produced or stock components. The underlyingform, size, shape, or other properties of the key components must becustomized to provide a truly unique and custom product for the user.Once having been able to obtain the user's image data, it is thendesirable to analyze and make critical measurements of the user's face,determine user preference, and on-demand manufacture a custom piece ofeyewear.

It is of course desirable for the process to be as automatic as possibleand be one that returns to the user the most perfect one-of-a-kind pieceof eyewear that he or she has ever seen. If this can be done in arelatively swift fashion, the user is provided with a quick unique pieceof eyewear is that manufactured on demand.

More particularly, the online market is rapidly growing, though therestill persist numerous problems for consumers. Consumers have poorability to try-on glasses while shopping online. Online sites have moreselection than in stores, but often the consumer is faced with endlesspages of glasses from which to choose. The quality of the glasses isoften unknown, and consumers are even more concerned about their newglasses fitting correctly and being comfortable since they cannotphysically hold or see them until they purchase.

A clear need exists for a shopping experience that enables a uniquemade-to-order product with high quality materials and design, at a pricethat users believe is fair and affordable for a made from scratch uniqueone up item., and an easier and more custom experience to creating andpurchasing the perfect product for the individual, in this case a pairof glasses.

The concept of virtually trying on articles of clothing, includingeyewear, has been discussed in the prior art for a number of years. Allof the below listed patents relate to preview systems, but none relateto providing a from scratch product, relying instead on prefabricatedcomponents for a particular item.

For instance, Spackova in U.S. Pat. No. 4,539,585 describes a computersystem to view articles of clothing on a person in an image. Mori, U.S.Pat. No. 4,730,260, and Ninomiya, et.al. U.S. Pat. No. 4,845,641,describe computer systems to virtually overlay eyewear on a person in animage. Jordan, U.S. Pat. No. 5,280,570, describes a system requiring auser to visit a store to virtually try on glasses with a realisticrendering of how their eyes will appear behind the glasses. Norton, U.S.Pat. No. 5,592,248, describes various methods of overlaying virtualimages of eyewear on an image of a person's face to preview theappearance. Faye, U.S. Pat. No. 5,983,201, describes a system for usersto virtually try on a variety of eyeglasses on their personal computerby connecting to an online store, selecting a subset of eyewear based onuser preferences and sizes, and allowing user to purchase the frames.Gao, U.S. Pat. No. 6,095,650, describes another system for capturing animage and displaying eyewear superimposed on the user's image, includingscaling of the image and detection of pupils to center the frames.Saigo, U.S. Pat. No. 6,142,628, describes another try-on system thatalso includes lens selection and display of lens shape in addition toframes. Waupotitsh, U.S. Pat. No. 7,016,824, describes an eyewearpreview system that used a 3D face model provided by the user to overlayeyewear models on. Abitbol, US 6,692,127, describes an eyewear try-onsystem that requires a wide-view camera to obtain a 3D model. Foley,U.S. Pat. No. 6,535,223, describes a system to determine pupillarydistance based on an image of a person's face including an object of aknown scale, as well as superimposing preview eyewear and allowingorders to be placed.

All of the previously described prior art explore various ways ofpreviewing eyewear superimposed over an image of a person, but they arenot on-demand systems that create, assemble and deliver a uniqueone-of-a-kind product from scratch. Nor do they permit previewing newcustom eyewear that has not previously been mass-produced. Nor do theyuse the user-specific information to make eyewear better for the user.In short., they do not customize, adapt, modify, implement, or createnew products such as eyewear using an on- demand system providingone-of-a-kind products from scratch. Moreover, all of the abovetechniques rely on previewing eyewear superimposed on the image of aperson.

On the other hand, Fujie, U.S. Pat. No. 5,576,778, describes a system todesign eyewear based on facial dimensions of a person. It is noted thatFujie is limited to controlling various anchor points on a Bezier curvethat is extracted from facial image data to achieve a design. However,the specification of these anchor points or the control thereof by anindividual is technical and difficult, made more so because these pointsare controlled using the user's words to control shape. Moreover, Fujieis limited to specifically sending polar coordinates based on Beziercurves to machine tools. This is much too complicated for a user, andthe user's words alone may not be suitable as the only control.

Soatto, U.S. Pat. No. 6,944,327, describes a system to customize eyewearbased on preview images of the user's face. However, Soatto does nottake into account automatically-generated user preferences. Soatto doesnot describe an on-demand end-to-end process and does not describe afull system that can actually manufacture eyewear. Moreover, the Soattomethod is limited to specific cameras, only a frontal face image andusing a method to generate a two dimensional template of the face forsizing. Limiting the preview to only a front image prevents sizinginformation that is critical around the temples for ensuring a goodpreview and comfort for the user. Moreover, most computer systems do nothave multi-lens cameras conveniently available. Note, adjustment is doneonly through control points while maintaining a constant perimeter rimsize, which is of limited application—different users will surelyrequire different sizes. It will be appreciated that methods describinga 3D model of the face require two or more cameras not normallyavailable to most users.

Izumitani, U.S. Pat. No. 6,533,418, describes a system to make eyewearto order based on image previews superimposed over the user's face.However this patent only discusses changing lens shape, frame types,frame parts, and colors. It does not explain changing frame shape, butonly replacing parts or changing a frame style from rimless to rimmed,which is very limiting when one wants to more fully customize eyewear.Moreover this patent does not describe automatic algorithms that size aframe to a user's face or aid in the selection of the best frames.Instead it uses a manual system like a custom order catalogue with manyinterchangeable parts to choose from, which could be overwhelming or toocomplicated for an eyewear consumer. Additionally, the preview systemdescribed only shows front and side portraits of the user with eyewear,with no interactive views, 3D views, or video, and it does not measurethe dimensions of a face automatically. Further, a user is required toassist or enter information to obtain proper measurements. Finally,while the patent describes the manufacturing of eyewear, it does notclearly describe how made-to-order eyewear could actually be produced.

Warden, U.S. Pat. No. 7,845,797, describes a method for manufacturingcustom eyewear that uses a front and side image in a system withmultiple cameras and lighting sources. The method requires the captureof images with and without eyewear worn on the user's face before itdetermines the best lens position. This method is quite limited, as itrequires that the user already physically possesses the eyewear hedesires, and it assumes the user simply wishes to refine the lensplacement in a subsequent pair of frames. In short, this is not anon-demand end-to-end system that starts from scratch to then create,design, assemble and deliver the custom product.

To satisfy the needs of a typical consumer, an easy-to-use method andsystem that can provide a confident and enjoyable shopping experienceare necessary. The system must be capable of working with the computerhardware and image capturing equipment available to typical consumer,which limits the minimum hardware to a single-lens digital camera, standalone or embedded in a computer system, without depth ordistance-measuring capability. The embodiments of this inventiondescribe both systems to use single camera hardware and also systemsthat benefit from multi-camera or depth camera technology, in the eventthese technologies become more pervasive in a form used by consumers orin the event that a computer system is installed in a retail or officelocation.

The prior art describes technologies that are designed mostly for theaesthetic preview of the eyewear on a user. A need for a morequantitative analysis exists to enable a better experience, custom fit,custom style, automated adjustment and recommendations, and the overallability to make an eyewear design fit with each user's unique anatomyand taste.

Often pupillary distance is the only measurement taken to ensure theproper fit of eyewear, and that measurement alone is not sufficient toensure a proper physical fitting of custom eyewear. More information isespecially needed for advanced optics, such as progressive ordigitally-compensated or freeform lenses. But regardless of the type andquantity of facial measurements needed to craft custom eyewear, the usershould not be required to manually measure them. Most target users arenot technologically savvy beyond following easy prompts in a webbrowser. A consumer needs an experience that is easier than picking andchoosing parts and pieces or custom drawing every detail, especiallywhen using only 2-D images, as the prior art has described. The methodand system must enable easy customization, including automation ofsizing and styles if the user desires automated recommendations. Anaverage user should be able to obtain any eyewear design they desire andan excellent fit by having a design custom-fitted to his face, seeing apreview in a “what you see is what you get” display, and being able tomake changes and see the effect on his face and fit.

Finally, the method and system must result in a manufacturable product,such that it can be produced and sold at a reasonable cost to the userwith an acceptable delivery time. It will be appreciated that a greatpreview system is not useful if the product being previewed is notultimately manufacturable at a cost and in a time frame that issatisfactory to the user ordering the product.

Thus there is a compelling need for a method and system to allow greaterand more personalized customization of lenses and frames, more accuratemodeling and preview, more automated or assisted eyewear selection andcustomization, more detailed measurements, and methods to producecustomized eyewear efficiently and economically to fulfill users'orders.

SUMMARY OF THE INVENTION

The subject invention has a number of important parts. The first part isthe understanding that what is desired is a from-scratch, one-upcustomized product that is not manufactured exclusively fromoff-the-shelf, previously designed, mass-produced, or stock components.As mentioned above, there are many systems which involve picking anumber of components that are premade or pre-manufactured and puttingthem together in a customized object. However, if there are a lot ofmass-produced items, the user does not have the feeling that he or sheis presented with a truly unique one-off product centered on theparticular profile of the user. Nor will a product made frommass-produced parts be customized to the desired degree needed to fitthe user's unique anatomy and preferences. One must create at least somepart of the custom product completely from scratch to fit the user, forexample making some form of the product into a unique, non-mass-producedshape or size. The ability to automatically design and alter thefundamental shape and form of a custom product, with or without userguidance, is an important advantage over systems that simply let usersbrowse and assembly mass-produced components.

The second part is how one ascertains the anatomic features of theindividual, what one measures when measuring the anatomical features,and how one utilizes these anatomic measure features in the creation ofa one-up from scratch object.

The third part is to be able to ascertain a user's profile, his habitualbuying habits, his likes and dislikes, derived over a period of time andto be able to use all of these likes and dislikes and profiles toprovide for the user a suggested unique product.

Fourthly, taking all of the above information into account with aproduct having been modeled after the user's anatomic features andpreferences, it is important to be able to manufacture a unique producton-the-fly and deliver the product to a user in an acceptable timeline.The output being a unique product the user may have thought aboutwanting or which he or she may have never thought about, but is providedwith due to the predictive nature of the process flow that results inthe on-demand product manufacture.

Thus, at a high level, the subject system is an end-to-end system thatenables a user to obtain a completely custom product from scratchwithout the limitation of exclusively using off-the-shelf, previouslydesigned, mass-produced, or stock components. The product ismade-to-order and best suited to the user's anatomy and personalpreferences. The system may integrate steps from acquiring data aboutthe user through delivering the final product. This goes well beyond theprior art by offering innovations that permit design and fabricationfrom the start without using exclusively stock, predesigned, orprefabricated parts. Rather the product is designed ab initio andautomatically utilizing some or all of the following: the user's likesand dislikes, his unique anatomical attributes and unique requirementsso that the finished product in terms of design, shape, fit, size,color, weight, finish, function, and artistic impression will be asclose as possible to the user's wishes. Additionally, since the systemmay be considered an expert system, it is like providing a user with aspecialist in order to provide a product with the most appropriate styleand fit. The subject system, suggesting choices at every turn reflectsthe so-called artificial intelligence of the expert.

Not only is the system itself unique, but various techniques aredescribed in order to develop anatomic models, directly derive certainanatomic features, various imaging techniques, ranging and sizecharacterization techniques, scaling techniques, product presentationtechniques, user interaction techniques, and custom manufacturingtechniques; these techniques add to the already unique features of thesubject system.

One of the features of the subject invention is the ability to obtainthe features of an individual and more particularly his or her face. Itis been found that self portraits, for instance done through theutilization of smart phones or electronic cameras can be useful inproviding the image information necessary for the deriving the requiredanatomic models. Even though the so-called “selfie” or self-portraitfrom a camera phone is not three-dimensional, various features of theimage formed from the smart phone can be utilized in generating 3-Dmodeling of a person's face. Thus, a convenient method of inputting aperson's anatomical features, is to use the ubiquitous cell phone forthe image capture, it being a finding of the subject invention thatthere is sufficient information in the self-portrait from a singlecamera to permit anatomical modeling.

While the subject invention will be described in connection witheyewear, it is within the scope of the subject invention to design,manufacture and deliver from scratch personalized products of anynature, for instance including jewelry, clothing, helmets, headphones,and other personal items. The scope also focuses on one-up, customproducts made from scratch, but the methods described could also beapplied to highly unique custom products that are not necessarily 100%one-up or made from scratch. Many products would benefit from having ahigh variety of designs to provide custom products (e.g. hundreds,thousands, millions of designs), which are too difficult to configure,stock, or manufacture using traditional methods and would be highlysuitable to the methods described herein. A high degree ofconfigurability that requires a product to be custom made-to-order iswithin the scope of the invention.

The comprehensiveness of the subject on-demand end-to-end system relieson the following:

Obtaining and Analyzing Image Data and Anatomic Information

In the subject invention, new methods that enable improved oralternative ways to achieve capturing images and determining anatomicinformation and models of the user. These include more detailed anatomicdata, aesthetic analysis, and other metrics, which are used to informboth eyewear frames as well as advanced optical designs. Heretoforethere has been no attempt to use anatomic information, aestheticinformation, and other metrics extracted from image data to inform suchdetailed designs.

Obtaining Other User Information

Other user information and preferences, not obtained automatically fromimage data, may be used to provide further information to customizeproducts. This information is used in novel prediction and learningalgorithms that enable a product design to be altered to suit aparticular user.

Configurable Product Models

The subject invention describes configurable product models that enablecustomization that is far more personalized than interchanging stockcomponents to make a custom assembly. The configurable models allowentire shapes, contours, 3D surfaces, measurements, colors, finishes,and more to be completely customized for an individual user.

Product Customization

Algorithms are used that customize the shape and style of eyewearautomatically for the user based on their anatomy derived from the imagedata that is analyzed as well as personal preferences. Alsoprognostication algorithms are used to predict user taste and design toassist in the custom product design and fabrication. This helps presentthe user with the highest probability designs upfront.

Previewing One-Up Custom Products to the User

The subject methods offer high-fidelity renderings of one-up customproducts. These are not standard previews of previously existingproducts. The preview of one-up custom products, such as eyewear, occurprior to the product ever being produced or existing since it is madespecifically and uniquely for the user. These previews involve moreadvanced techniques than previews of existing products because theproduct has not existed and prior photos, documentation or testing ofthe product representation does not exist yet. Everything must begenerated or configured on-the-fly to enable a high quality preview of aone-up custom product that has not been built yet. The subject system isnot merely rendering existing products (e.g. eyewear or parts ofeyewear), but provides completely new custom designs from scratch.

User Interaction with Product Preview

Various improved methods allow the user to interact with custom productpreviews, alter custom designs in real-time, get feedback from others,and allow other friends/designers/opticians to also design customproducts for them.

Manufacturing Custom Product

Unlike the prior art that describes very basic methods of customization,such as interchanging parts or limited customizing some components ofeyewear, the subject system produces completely custom products, such aspremium eyewear, from scratch. The one-up custom eyewear includes framesand lenses, built to order in a specific shape, size, and color for oneuser. The subject system is using advanced techniques that allow eyewearto be delivered with the same high-quality materials and finish ofregular premium eyewear, but with completely custom designs.

Shopping System

Finally, the subject invention includes a shopping system that enablesthe user to progress through the steps necessary to obtain customproducts, input their data and preferences, and select and purchase theproduct.

Definitions

The following definitions are for explanatory purposes to help definethe breadth of words used herein. These definitions do not limit thescope of the invention, and those skilled in the art will recognize thatadditional definitions may be applied to each category. By way ofdefinition as used herein, image data includes 2D image(s), digitalimages, video, series of images, stereoscopic images, 3D images, imagesacquired with standard light-sensitive cameras, images acquired withcameras that have multiple lenses, images acquired with depth cameras,images acquired with laser, infrared, or other sensor modalities.Computer systems include tablets, phones, desktops, laptops, kiosks,servers, wearable computers, network computers, distributed or parallelcomputers, or virtual computers. Imaging devices include single lenscameras, multiple lens cameras, depth cameras, laser cameras, infraredcameras, or digital cameras. Input devices include touchscreens, gesturesensors, keyboards, mouses, depth cameras, audio speech recognition, andwearable devices. Displays include panels, LCDs, projectors, 3Ddisplays, heads-up displays, flexible displays, television, holographicdisplays, wearable displays, or other display technologies. Previewedimages in the form of images, video, or interactive renderings includesimages of the user superimposed with product model images, images of theuser superimposed with rendering of product model, images of theanatomic and product models of the user. Anatomic models, details, anddimensions include length of features (eg length of finger), distancebetween features (eg distance between ears), angles, surface area offeatures, volume or features, 2D contours of features (eg outline ofwrist), 3D models of features (eg surface of nose or ear), 3Dcoordinates, 3D mesh or surface representations, shape estimates ormodels, curvature measurements, or estimates of skin or hair colordefinition. A model or 3D model includes a point-cloud, parametricmodel, a texture-mapped model, surface or volume mesh, or othercollection of points, lines, and geometric elements representing anobject. Manufacturing instructions include step-by-step manufacturinginstructions, assembly instructions, ordering specifications, CAM files,g-code, automated software instructions, co-ordinates for controllingmachinery, templates, images, drawings, material specifications,inspection dimensions or requirements. A manufacturing system includes acomputer system configured to deliver manufacturing instructions tousers and/or machines, a networked computer system that includesmachines configured to follow manufacturing instructions, a series ofcomputer systems and machines that instructions are sequentially passedthrough. Eyewear includes eyeglass frames, sunglass frames, frames andlenses together, prescription eyewear, non-prescription (piano) eyewear,sports eyewear, or electronic or wearable technology eyewear.

Custom Products

The following is an embodiment for product that is custom fit anddesigned based on user anatomy derived from image data, previewed,altered by user preferences, and then manufactured to order for thefirst time after customization:

In accordance with an embodiment, methods are disclosed for creatingcustom products. One method includes acquiring, using at least onecomputer system, image data of a user; determining, using at least onecomputer system, anatomic details and/or dimensions of the user;configuring (eg, custom shape, size, dimensions, colors, finish, etc),using at least one computer system and anatomic data of the user, a newproduct model for the user; applying, using at least one computersystem, a configurable product model to the image data or anatomic modelof the user; previewing, using at least one computer system, images ofthe user with the configurable product model; optionally adjusting andupdating the preview, using at least one computer system and/or userinput, the configurable product model properties (eg, custom shape,size, dimensions, colors, finish, etc); preparing, using at least acomputer system that executes instructions for manufacturing the customproduct based on the previewed model; and manufacturing, using at leastone computer system and manufacturing system, the new custom product.

In accordance with an embodiment, systems are disclosed for creating acustom product. One system includes an image acquisition deviceconfigured to obtain image data of a user; an input device configured toreceive instructions from a user; a display configured to display imagedata to a user; a manufacturing system configured to produce a customproduct; a digital storage device to store instructions for creating andpreviewing custom product; a processor configured to execute theinstructions to perform the method including: acquiring, using at leastone computer system, image data of a user; determining, using at leastone computer system, anatomic details and/or dimensions of the user;configuring (eg, custom shape, size, dimensions, colors, finish, etc),using at least one computer system and anatomic data of the user, a newproduct model for the user; applying, using at least one computersystem, a configurable product model to the image data or anatomic modelof the user; previewing, using at least one computer system, images ofthe user with the configurable product model; optionally adjusting andupdating the preview, using at least one computer system and/or userinput, the configurable product model properties (eg, custom shape,size, dimensions, colors, finish, etc); preparing, using at leastcomputer system, instructions for manufacturing the custom product basedon the previewed model; and manufacturing, using at least one computersystem and manufacturing system, the new custom product.

Systems are disclosed for creating a custom product. One system includesan image acquisition device configured to obtain image data of a user;an input device configured to receive instructions from a user; adisplay configured to display image data to a user; a manufacturingsystem configured to produce a custom product; a digital storage deviceto store instructions for creating and previewing a custom product; and,a processor configured to execute the instructions to perform themethod.

The system includes acquiring the image data of a user; determininganatomic details and/or dimensions of the user; configuring the productto take into account these details by providing a corresponding newproduct model; applying a configurable product model to the image dataor anatomic model of the user; previewing images of the user with theconfigurable product model; optionally adjusting and updating thepreview; preparing, instructions for manufacturing the custom productbased on the previewed model; and manufacturing the new custom product.The above can be accomplished using a properly programmed computer orcan be in the form of a non-transitory computer readable medium.

More particularly, a system and method are disclosed for creating customeyewear including at least one computer system configured to receiveimage data of a user. The computer system is further configured toreceive other data from the user, including but not limited todemographics, prescription, preferences, etc. The system and method mayinclude determination of quantitative anatomic information regarding theuser from the user-provided data. The system and method may includecustomization of the properties of an eyewear model, including size,shape, color, finish, and style, to satisfy the anatomic and style needsof the user. The system also includes physically manufacturing thecustomized eyewear such that it matches the previewed representation.

In accordance with an embodiment, a system and method are disclosed forcreating and visualizing custom eyewear including at least one computersystem configured with a display. The computer system is furtherconfigured with at least one image capture device to capture image dataand/or measurement data of a user. The computer system is furtherconfigured to receive other data from the user, including demographics,prescription, and preferences. The system and method may includedetermination of quantitative anatomic information regarding the userfrom the user-provided data. The system and method may includevisualization of an eyewear model superimposed on the user's image datain the proper position on the user's face. The system and method mayalso include customization of the properties of the eyewear model andproviding an updated preview of the customized eyewear superimposed onthe user's image data. The system and method includes physicallymanufacturing the customized eyewear such that it matches the previewedrepresentation.

In accordance with another embodiment, a system and method are disclosedfor automatically customizing eyewear. The computer system is furtherconfigured to analyze the user's image data, quantitative anatomicinformation, and other provided data to determine optimal properties forthe eyewear model such that it best matches the user's anatomy and stylepreferences.

In accordance with another embodiment, a system and method are disclosedfor interacting with a custom eyewear model. The computer system isfurther configured with an interface application. The system and methodmay include obtaining input or commands from a user through the computersystem. The system and method may further include controlling thevisualization, including angle, zoom, and rotation of the eyewearpreview. The system and method may further include controlling theposition and orientation of the eyewear model of the user's image data.The system and method may further include enabling the user to directlycustomize the properties of the eyewear model and provide an updatedpreview.

In accordance with another embodiment, a system and method are disclosedfor automatically defining optical lens designs. The system and methodinclude analyzing the user's quantitative anatomic information,prescription information, and custom eyewear model to calculateparameters needed to inform optical design, including interpupilarydistance, vertex distance, face wrap, eyewear and frame outline. Thesystem and method are further configured to provide the parameters to amanufacturing system for the design and manufacture of custom lenses.

In accordance with another embodiment, a system and method are disclosedfor a web interface for purchasing custom eyewear. The computer systemis further configured with a data transfer means The system and methodinclude providing an interface for a user to select eyewear designs,interact with, preview and customize eyewear designs, order eyewear, andtransfer all information needed to build and ship custom eyewear to theuser.

In accordance with another embodiment, a system and method are disclosedfor controlling manufacturing of custom eyewear. The computer system isfurther configured to transfer data and information to at least onemanufacturing system .The system and method include transferring customeyewear models or parameters, user information, and an order to themanufacturing system. The system and method further include convertingthe eyewear model or parameters into manufacturing data used to controlmanufacturing equipment. The system and method also include providinginstructions for machinery, robotics, and human operators to build,inspect, and ship custom eyewear.

In accordance with another embodiment, a system and method are disclosedfor a parametric eyewear model. The system and method include arepresentation of eyewear that contains dimensional informationregarding the shape and size of the eyewear design. The system andmethod further include parameters that define certain key features ofthe eyewear model, including but not limited to length, width, height,thickness, and radii. The system and method further include the eyewearmodel updating when at least one parameter is changed, automaticallyaltering the eyewear to satisfy the constraints of all parameters.

In accordance with another embodiment, a system and method are disclosedfor learning from a user's interactions and preferences involving alearning machine or predictor or prognostication machine. The system andmethod include tracking the actions a user takes selecting, customizing,and previewing eyewear. The system and method further include machinelearning analysis of the tracked actions in addition to the userprovided image data, quantitative anatomic information, and otherprovided information to determine user preferences for custom eyewearproperties. The system and method further include making recommendationsto the user based on the learning analysis.

In accordance with another embodiment, a system and method are disclosedfor learning from a body of data. The system and method include buildinga database of image data, quantitative anatomic information,preferences, and other information relating custom eyewear to userinformation. The system and method include training machine learningclassifiers to predict the preference of a user based on their data. Thesystem and method further include applying the analysis to a new user tobest provide a custom eyewear design that will suite the user's anatomyand preferences.

In accordance with another embodiment, a system and method are disclosedfor guiding the user through a customization process. The system andmethod include providing a sequence of instructions or questions toguide the user through the steps needed to customize eyewear for theirpreferences and anatomy.

In accordance with another embodiment, a system and method are disclosedfor prediction of a poor fit. The system and method include analyzingthe fit between the user's quantitative anatomic information and acustom eyewear design. The system and method include using simulation,physical modeling, and analysis to predict when a sub-optical fitbetween the eyewear and user is designed. The system and method furtherinclude informing the user of the sub-optimal design or automaticallycorrecting it.

In accordance with another embodiment, a system and method are disclosedfor previewing vision through a customized eyewear model. The system andmethod include rendering a preview of the vision through a customeyewear model, including the shape, size, and optical properties of alens. The system and method include rendering a live or static scenethat simulates the user's vision, including but not limited todistortion, area of focus, color, and other optical effects.

In accordance with another embodiment, a system and method are disclosedfor copying another pair of eyewear. The system and method includereceiving image data of a person, including the user, wearing eyewear.The system and method further include detecting the eyewear andanalyzing the shape, color, and size. The system and method furtherinclude optimizing a custom eyewear design to match the analysis of theshape, size, and color. The system and method further include previewingthe custom eyewear on the user's image data and allowing furthercustomization.

In accordance with another embodiment, a system and method are disclosedfor sharing custom eyewear previews and the ability to customizeeyewear. The system and method include sending permission from at leastone computer system to at least one other computer system to preview andcustomize eyewear on a user's image data. The system and method furtherinclude allowing a third party to interact with, customize, and updateeyewear models on the user's image data. The system and method furtherinclude the third party to provide feedback and updated designs to theuser.

In accordance with another embodiment, a system and method are disclosedfor matching eyewear color to another object. The system and methodinclude obtaining image data or information (including but not limitedto manufacturer, part number, etc) about an object with a desired color.The system and method further include calibrating the color of the imagedata with a reference image. The system and method further includeextracting the color properties of the desired object and applying thecolor to the custom eyewear model.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features of the subject invention will be betterunderstood in connection with the Detailed Description in conjunctionwith the Drawings of which:

FIG. 1A is a block diagram of a system to create a from-scratch one-upcustomized product without the exclusive use of off-the-shelfcomponents;

FIG. 1B is a block diagram of a custom eyewear shopping system;

FIG. 2 is a block diagram an image capture portion of the subject systemshowing the interplay between an image capture device, user inputs, andother information coupled to a computer system which drives themanufacturing process;

FIG. 3 is a diagrammatic illustration of eyewear and eyewear parts whichcan be customized through the use of the subject system;

FIG. 4 is a diagrammatic illustration of a user's face and anatomicfeatures;

FIG. 5 is a diagrammatic illustration of a computer system to captureimage data;

FIG. 6 is a diagrammatic illustration of dimensions between a face andeyewear for analyzing a face, thereby to permit further facial andeyewear parameters;

FIG. 7 is a diagrammatic illustration of additional dimensions of facesand eyewear;

FIG. 8 is a diagrammatic representation of a parameterized quantitativeanatomic model;

FIG. 9 is a diagrammatic illustration of an example of a parameterizedeyewear model before and after adjustment to custom fit width withoutaffecting other key dimensions;

FIG. 10 is a diagrammatic illustration of two eyewear designs withoptimal eye center locations;

FIG. 11 is a diagrammatic illustration of an example computer systeminterface for previewing, correcting, and customizing eyewear;

FIG. 12 is a diagrammatic illustration of an example illustrationshowing the custom adjustment of the width of eyewear with a computersystem an interface to be able to ascertain the product placement on theface of the individual as well as improvements that can be made at thetime that improve representations of the individual;

FIG. 13 is a diagrammatic illustration of an example illustrationshowing an eyewear design being edited;

FIG. 14 is a diagrammatic illustration of an example of automatedeyewear model adjustment to optimize parameters;

FIG. 15 is a diagrammatic illustration of an example of a custom 3Deyewear model converted to flat patterns for manufacturing;

FIG. 16 is a diagrammatic illustration of an example of a custom 3Deyewear model and manufactured part;

FIG. 17 is a diagrammatic illustration of a computer with an imagingdevice to acquire the image of the user utilizing a reference;

FIG. 18 is a diagrammatic illustration of a computer system toco-register an anatomic model with an original user image;

FIG. 19 is a diagrammatic illustration of the use of the computer systemto reconstruct a model of a user's face and a model of a referencetarget based on image data;

FIG. 20 is a diagrammatic illustration of the scaling of an anatomicalmodel to a user's face using a double mirror reflection system;

FIG. 21 is a diagrammatic illustration of the building and scaling of ananatomic model of the user's face from a collection of previouslyacquired images and fitting a 3-D face model across feature sets andcamera positions;

FIG. 22 is a diagrammatic illustration of the scaling of a user's faceusing existing eyewear already possessed by the user;

FIG. 23 is a diagrammatic illustration of a system for measuringdimensions of a reference object by displaying a reference box andcalculating pixel size and the true size of the reference box;

FIG. 24 is a diagrammatic illustration of a system for customizingeyewear design optimized to fit asymmetric facial features;

FIG. 25 is a diagrammatic illustration of a system to achieve asimulated camera perspective;

FIG. 26 is a block diagram of an in-store custom eyewear shoppingmethod;

FIG. 27 is a block diagram of an in-store custom eyewear shoppingsystem;

FIG. 28 is a diagrammatic illustration of a system for customizingeyewear nosepads to fit different users' anatomies;

FIG. 29 is a diagrammatic illustration of configuring a custom productmodel, demonstrating a small portion of the degree of shape and sizecustomization;

FIG. 30 is a diagrammatic illustration of customizing an eyewear modelprior to aligning it to an anatomic model;

FIG. 31 is diagrammatic illustration of customizing an eyewear modelafter aligning it to an anatomic model;

FIG. 32 is a block diagram of a manufacturing sequence for custom one-upproducts; and

FIG. 33 is a diagrammatic illustration of creation of a custom helmet.

DETAILED DESCRIPTION

Referring to FIG. 1A, a system is provided in which a computer system 14creates a custom product from scratch based on inputs to the computersystem, including input based on the user image. From scratch refers tothe fact that what is provided is a one-up customized product that ismanufactured without the exclusive use of off-the-shelf, previouslydesigned, previously produced, or stock components. This does not meanthat incidental components such as fasteners, hinges, and the likecannot be available as parts of a custom product. However, the majorcomponents of the product are designed ab initio, thus to give theproduct a new type of uniqueness, unlike that available by productswhich are assembled from pre-manufactured components.

It is important to understand where the computer system that generatesthese custom products obtains information. The computer system obtainsimaging data of the user, determines anatomic data, measurements fromimage data, and further optional user preferences and information suchas the users likes or dislikes, ascertained from analysis of the userscomputer history. The computer system also accepts inputs from the user,where the user may specify certain preferences or directly control someaspects of the product customization.

The system does not operated in a vacuum; in other words, the computersystem does not generate custom products from nothing. In order for thecomputer to start its creative process, configurable product models areinstalled on the computer system that at least specify in some broadoutline, structures and specifications that are necessary for thecustomizable product.

With this having been said, and as illustrated at 10, computer system 14obtains and analyzes image data and determines a user's anatomicmeasurements and details. As has been noted hereinbefore, image capturecan be accomplished in a variety of different ways, most notably byutilization of a self-portrait generated from a handheld electronicdevice such as a smart phone or electronic camera. This is a convenientimage capture method for the average user who may utilize the ubiquitouscell phone as the point of departure for defining his or her ownanatomical features.

The computer system as illustrated at 12, obtains optional userpreferences and information which may be gleaned from a wide variety ofsources. The computer system at 14 is provided with at least oneconfigurable product model 13 to guide the computer system. Havinganalyzed all of its inputs, computer system 14 automatically outputs anew custom product model. The output of the computer system 14 istherefore provided to preview system 15 in which the computer systemcreates previews of custom products and the user. Then, as illustratedat 17, the computer system prepares product models and information formanufacturing the selected one-up, fully-custom product.

Note that it 16, optional user interaction is provided to update,inform, or control the preview, and custom products. After the computersystem has created previews of custom product, the user may specifyoptional user interaction to update, inform, or control the preview, andcustom products. When these addition control instructions are input tothe computer system 14, the system is able to carry out the optional newdirections for the custom product, either directly incorporating userchanges or using input to inform new custom product models.

More particularly, the system operates as follows. The computer systemobtains the image data at 10 by a variety of means, such as a camera orimaging device connected to the computer system, with image datatransferred to the computer system by the user, or image datatransferred from another computer system. The anatomic measurements anddetails may result in dimensions, models, shape analysis, etc, and willbe described in further detail.

As illustrated at 12, computer system 14 obtains other optional userinformation and preferences. This information, such as demographicinformation, medical or prescription information, answers to questions,style choices, keywords, etc may be used as further inputs to thecomputer system's automatic analysis and customization of a product forthe user.

As illustrated at 13, the computer system contains configurable productmodels added by the manufacturer or designer. These configurable productmodels are representations of the custom product, and they may bemodified to alter properties including shape, size, color, finish, etc.The configurable models may have thousands, millions, or infinitevariation, yet they are also created with the ability to constrain orrestrict configurability to a domain that the manufacturer chooses (e.g.only a certain range of material thicknesses may be used or certaindimensions must not change when others are configured). The configurablemodels may contain sub-components, such as fasteners, that aremass-produced or pre-designed, but the major custom components whenassembled with the sub-components results in a highly customized,one-up, from scratch product.

As illustrated at 14, the computer system uses the inputs consisting ofthe configurable product model, user image data, user anatomic data, andoptional user preferences to generate a new custom product model. Thecomputer system may use a variety of techniques, including equations,analytics, shape models, machine learning, clustering, lookup tables,etc to produce a final custom product model. The computer system mayalso produce a range of custom models for the user to choose from. Thesecustom models are considered one-up, non-stock, and completely customfor the individual user.

As illustrated at 15, the computer system creates a preview of thecustom product model. The preview may consist of images of the customproduct, renderings of the custom product model on the user's anatomicmodel, renderings of the custom product model on the user's image data,physical rapid prototypes of the custom product model, etc. The previewsmay be shown to the user on a display of the computer system.

As illustrated at 16, the computer system accepts user input to update,inform, or control the custom product model. The user, or others givenpermission by the user, may change the preview, select configurableoptions of the custom product model such as color or size, answerquestions to refine the product model, or the user may directly alterthe configurable model to their preferences (i.e. changing the shape orstyle).

As illustrated at 17, the computer system prepares the custom productapproved by the user for manufacturing. Preparation may involveconverting the custom product model and user preferences to a set ofspecifications, instructions, data-structures,computer-numerical-control instructions, 2D or 3D model files that canbe interpreted by manufacturing systems, etc. Preparation may alsoinclude custom computer-controlled instructions for guiding machinery orpeople through each step of the manufacturing process.

As illustrated at 18, the computer system provides instructions to amanufacturing system, which produces the one-up custom product. Variousspecific methods will be described for producing a one-up customproduct.

The previously mentioned computer and manufacturing system are describedgenerally in FIG. 2 as a block diagram of computer system 220 used by auser 200. In an exemplary embodiment, at least one computer system 220,including but not limited to a tablet, phone, desktop, laptop, kiosk, orwearable computer, is configured with a display 230 for presenting imagedata to a user. The display 230 includes LCD screens, flexible screens,projection, 3D displays, heads-up displays, or other displaytechnologies. The computer system 220 has an input device forcontrolling the computer system, included but not limited to atouchscreen, keyboard, mouse, track pad, or gesture sensor. The computersystem 220 is further configured with an image capture device 210,including but not limited to a single-lens camera, video camera,multi-lens camera, IR camera, laser scanner, interferometer, etc. Theimage capture device is henceforth referred to as “camera”. The computersystem 220 is further configured to connect to a network or othersystems for communicating and transferring data 240. The computer system220 is configured to connect to other computer system(s) 250, includingbut not limited to servers, remote computers, etc. The other computersystem(s) 250 is connected to or in control of the manufacturing system260. The computer system 220 is further configured to provide aninterface to the user 200 for viewing, customizing, shopping, andordering custom products.

In addition to the custom product system for creating custom productsbased on user image data, anatomy, and preferences, the subjectinvention describes shopping systems that allow a user to gain access tothe custom product system: a means to shop, order, browse, interact,provide payment, etc. One embodiment for a custom eyewear shoppingsystem, which is built around the custom product system, is described:

Custom Eyewear Shopping System

Referring to FIG. 1B, a system for ordering custom one-up eyewear thatis created from scratch is detailed. As illustrated at 101 a user uses acomputer system to view eyewear and selects at least one style to try.This first step is optional, and the user may view a plurality ofeyewear on the computer display and choose to preview any of a pluralityof eyewear. The user may select styles to try and preview at thebeginning of their shopping experience, prior to purchasing, or at anytime they choose. As illustrated at 102 the computer system instructsthe user how to acquire image data and reference information. Thecomputer system camera captures image data consisting of one or moreimages, videos, or live previews of the user, and the computer systemdisplay shows the image data through its display. As seen at 103 thecomputer system analyzes computer image data and builds an anatomicmodel registered to image data. Thereafter, as illustrated at 104 thecomputer system prompts a user for prescription data, personal data andother information, which may be optionally entered at a later step. Thisis followed as illustrated at 105 by the computer system analyzing theinput information: measurements, anatomic model, user preferences, andimage data. As illustrated at 106, the computer system automaticallyadjusts size and fit of eyewear for the user. Additionally, asillustrated at 107, the computer system may automatically recommendsshape, style, and color choices to a user. As illustrated at step 108,the computer system creates at least one new custom eyewear model withat least one component designed from scratch and automatically placesthe eyewear model on user image data. The computer system renders apreview the custom eyewear model, which may include lenses, asillustrated at 109. The rendering may include combinations of the userimage data and user anatomic model with the custom eyewear model, aspreviously described.

As illustrated at 110, the user may interact with the computer system toadjust at least one of the eyewear size, shape, position, style, color,finish and patterns, etc. The result is illustrated at 111 in which thecomputer system recommends if the eyewear may not fit well or is notpossible to order based on the user interaction.

Thereafter as illustrated 112 the computer system stores the data andcalculates price and delivery estimates and any other relevantinformation the customer needs to decide whether to place an order ornot. As illustrated 113 the user may select alternate eyewear or theuser selects the custom eyewear to order as illustrated 114.

If the user selects alternate eyewear as illustrated at 113 the computersystem automatically generates a new custom eyewear model as illustratedat 108 and the process begins again.

Once the user selects the eyewear for an order, as illustrated at 114the computer system analyzes user information and models and preparesmanufacturing instructions, and as illustrated at 115 the computersystem prepares custom manufacturing files for the manufacturingequipment. Thereafter the computer system manages the manufacturingequipment and personnel to build the custom eyewear as illustrated 116.Finally the eyewear is shipped to the user as illustrated at 117. Thiscompletes the custom eyewear product, which was created and manufacturedfrom scratch for the user.

The following sections will describe further detail of the key stepsinvolved in creating a one-up custom product for a user:

Obtaining and Analyzing Image Data and Anatomic Information

The following section describes the detailed system and method forobtaining and analyzing image data and anatomic information, which isillustrated in FIGS. 1A at step 10 and 1B at 102, 103, and 105.

Before describing the detailed method for obtaining and analyzing imagedata and anatomic information, face anatomy and eyewear terminology aredescribed for reference. FIG. 3 shows eyewear 301, with various parts ofthe eyewear labeled. The front frame 302 holds the lenses 303 in place.The bridge 304 is in the center of the front frame 302, and the nosepads 305 extend off the front frame 302 to hold the eyewear 301 on thenose of the wearer. The hinges 306 connect the front frame 302 to thetemples 307, which rest on the tops of the wearer's ears at feature 308.FIG. 3 represents only one eyewear design, and it should be recognizedthat these basic parts may apply to other eyewear designs, or that someeyewear designs may have different parts.

FIG. 4 shows a user's face 401, eye 402, pupil 403 at the center of eye402, and eyebrow 404. The ear 405 also has a location denoted as the topof ear 406, where the temple of the eyewear would rest. The nose 407 isessential for support of eyewear. Cheekbones 408, mouth 409, forehead410, chin/jaw 411, nostril 412, and hair 413 are other features ofimportance in detecting and analyzing quantitative anatomic models.

Acquisition of Image Data

FIG. 5 shows a user 501 using a computer device 502 to acquire imagedata of their face 503. Instructions are provided to the user to placetheir face in certain positions while the computer system captures andanalyzes image data of the user's face. The computer system may utilizea smart phone or handheld electronic camera for the capture of the imageof the person's face. As mentioned hereinbefore, there is sufficientinformation from a single camera view of an individual to permit 3Dmodeling, and more particularly the generation of an anatomic model.

The computer system may require that certain objects are present in theimage to provide reference of scale. It is important to ensure thedimensions of the eyewear are appropriately sized relative to the user'sface, and providing dimensions to the image data or the resultinganatomic model and measurements is needed to ensure accurate sizing.Reference objects may include but are not limited to: coins, rulers,sheets of paper, credit cards, computer disks, electrical or computerconnectors, stamps, a calibration target on a computer device, or thecomputer device itself. The objects, when positioned near the user'sface, provide a reference dimension for the system to set dimensions tothe image data. If other image technology is available, such as a depthcamera, or if shape model techniques with intrinsic dimensions are usedthen reference objects may not be needed since the scale of the imagedata could be determined by the imaging equipment or shape model.

In an exemplary embodiment, once the user has followed instructions andis positioned in front of the computer system's imaging device,acquisition and analysis of their data begin. A first reference image iscaptured with a reference object held by the user in the same field astheir face. The image data captured by the computer is analyzed by thecomputer system to detect the reference object and measure its size, forexample in pixels. The image data is further analyzed by the computersystem to detect one or more of a plurality of features, including butnot limited to pupils, eyes, nose, mouth, ears, face, eyebrows, hair,etc. In an exemplary embodiment, the user's pupils are detected, andlandmarks placed on the center of each pupil. In another embodiment, theuser may optionally be queried to confirm or edit the location of eachpupil marker to ensure accuracy. With the data previously analyzed fromthe reference object the distance in pixels between pupils or otherfeatures is scaled from pixels to a unit of distance such as millimetersor inches. In another embodiment, the user may have previously acquireddata on a dimension(s) of their face, such as pupillary distanceobtained from an optometrist or an optical test, and the user may enterthis data into the computer system in lieu of using a reference objectfor scale. Alternatively, the reference image is acquired later in theprocess or at the same time as other image data acquisition.

The purpose of scaling the data with a reference object is to ensurethat measurements can be derived from the final quantitative anatomicmodel of the user. There are several key measurements to best determinehow to virtually place and fit eyewear on an image of a user's face.

FIG. 6 shows an illustration of the relationship between eyewear 601 anda user's face 602. The locations where the eyewear and face contact areof high importance since they control the fit of the eyewear. Thecontact locations between the eyewear 601 and the user's nose 603 areshown. Also shown are the contact locations between the eyewear 601 andthe user's ears 604, as well as the height and length between the top ofthe eyewear 605 and top of the ear 606.

As to FIG. 7, various detailed eyewear measurements are illustrated.FIG. 7 shows eyewear 701 with binocular interpupillary distance (Pd) 703a between pupils 702 and monocular interpupillary distance 703 b betweenthe center of the nose and pupil 702. Furthermore, if the highestquality optics are desired, or if specialized optics such as progressivelenses are desired, then additional measurements relating the eyes andoptics are useful, such as vertex distance 709 (distance from the eyesto the lens), pantoscopic tilt angle 710 (angle of the lens to the frontof the face), face or frame wrap 704 (curvature of frame around face),lens height 713 (vertical location of pupils in the lens), or opticalcenter. Prior art, as previously described, has been limited in notgenerating and using a wealth of information available from a fullquantitative anatomic model of a user's face in order to fully customizeeyewear frames and optics, as well as enable the best eyewear shoppinginterface and experience.

By way of example, FIG. 7 also shows the distance between nosepads ofthe eyewear 707. In this regard, FIG. 7 shows a model of a nose 711,which is used to derive quantitative measurements, including by notlimited to its length 712 and width 713 at various locations. Since eachuser's nose varies in dimensions, there is a great advantage in beingable to precisely measure its size and shape and then custom fit eyewearto perfectly fit that anatomy. Optimum comfort of an eyewear's nose padspositioned on a user's nose is achieved if the two contact surfaces arealigned properly and mate such that there are no high pressure-pointsand if the eyewear is naturally supported in the proper position by thenose. Each user may have a unique preference as to where on his nose heprefers to wear his eyewear for maximum comfort, aesthetic, or utility.Also, nose structure/shapes vary considerably between ethnicities. Forexample, users of Asian descent have noses with a smaller and flatterbridge than the noses of Caucasians, and they often prefer glasses thatare designed specific to their population. However, a distinct advantageexists in not designing for a population, but rather designing for anindividual user and their unique anatomic structure. Understanding thequantitative anatomy of the nose allows a custom eyewear to sitprecisely on the nose where desired with maximum comfort, aesthetic, andutility achieved out-of-the box without need for subsequent adjustment,which is often performed by an optical professional. However, properadjustments post-hoc of eyewear features such as nose pads, particularlyon plastic frames, is impossible for many eyewear designs.

FIG. 7 also shows additional measurements of the length of the temples705 and distance between the temples 706 needed to achieve a fit withthe user's face. Further, the brow, cheekbones, length of nose, andwidth of the head may provide limitations of where eyewear could fit ona user's face. Other dimensions of the face, such as the shape of head,curvatures, the length, shape, and angle of the nose, and more is usedto help suggest the best eyewear style and shape for a particular user.The locations of the pupils relative to eyewear are important to ensuregood optical quality.

In an exemplary embodiment, the computer system instructs the user toposition and move their head while the camera captures a series ofimages, or video. The rotation is side-to-side, up and down, or acombination. The computer system instructs the user to move their headto precise locations or just request that they approximate a movementshown on to them on the display. In another embodiment, the user has ahandheld computer system and moves the camera around their head ratherthan rotating their head. In another embodiment, the user already hasimages or videos to upload to the system, or the user captures images orvideos with another imaging device and uploads them to the computersystem, in lieu of capturing these with the computer system.

The captured video may consist of a series of images of the user's faceat various angles making up a set of image data. The computer system mayperform analysis on the images immediately as they are captured toprovide feedback to the user if there is a problem or if insufficientimage quality, poses, or quantity of data is acquired.

In an exemplary embodiment, the computer system analyzes the image datato ensure the user's face remains approximately within the center of theframe within certain bounds. The computer system may run a facedetection algorithm on the image data to detect the boundary of the facewithin each image. If the computer system detects the face outside thebounds, interference or occlusion detected in front of the user's face,or excessive blur or other unacceptable acquisition artifacts, then theuser is provided with a warning and instructions on how to re-acquire anew set of image data. Additionally, the computer system crops orexcludes portions of the image data before performing more intensivecomputations on the remaining dataset in order to reduce computationand/or transmission time. For example, the computer system may crop anypart of the image that is outside of the bounds of the detected face. Inaddition to detecting the face, the computer system may estimate thepose of the face (degree of rotation). The pose is estimated by usingvarious face detector or classifier algorithms that are trained todetermine poses. With a pose estimate for each image, the computersystem determines if an adequate range of poses have been captured. Ifnot, the computer system may instruct the user to reacquire. Thecomputer system may also filter unnecessary images. For example, theremay be duplicate poses or a small number of unacceptable images thatfall below a threshold for quality. Rather than reject the entire set ofimages, the computer system may reject a certain number of unacceptableimages and only process the images that pass the quality threshold,which is based on the previously described metrics.

The computer system automatically, or with user input, identifies theprecise image capture device and subsequently uses that understanding ofits optics to correct for optical distortions or utilize knowledge ofthe lens' depth-of-field to better analyze the dataset. Depending on theimage capture device, the computer system also corrects for distortionsor imperfections, such as lens barrel distortion observed on wide-anglelenses. These corrections enable the image data acquired to bestrepresent the user.

Quantitative Anatomic Model

Referring back to FIG. 1A at 10 and FIG. 1B at 103, the method describesthe construction of a quantitative anatomic model of at least a portionof the user's face and head. Once a complete set of image data isacquired, the computer system analyzes the image data to construct aquantitative anatomic model of the user's face. Various techniques areused to construct the model, and in an exemplary embodiment aquantitative anatomic model is represented as a surface mesh made ofelements, including but not limited to polygons, curvilinear elements,etc.

FIG. 8 shows an example of a mesh 804. The resolution of the mesh isaltered based on curvature, location, and features on the face, etc. Forexample, the detailed locations around the eyes and nose are higherresolution than areas where less detail exists, such as the top of thehead. In an exemplary embodiment, the face mesh only models the frontand side face area, though in other embodiments it models the entirehead or any portion thereof that is necessary including smaller regionsof the face, such as the eyes and nose only. Alternative representationsinclude point clouds, distance maps, image volumes, or vectors.

In an exemplary embodiment, a generalized quantitative anatomic model isdistorted to fit the user's face. The model is parameterized andrepresented as a mesh, with various mesh points affected by adjustingparameters. FIG. 8 shows an example of a model 801, with mesh elements804. In this example, a parameter influences the length 803 of the mouthfeature 802. If the parameter influencing length 803 were adjusted, thenthe appropriate elements of the mouth would adjust coordinates in orderto match the parameter specified. Other models, such as a shape model,may have generalized parameters like principal components that do notcorrespond to particular features but allow the generalized anatomicmodel to be adapted to a plurality of different face sizes and shapes.

The computer system analyzes the image data to iteratively perform asequence of feature detection, pose estimation, alignment, and modelparameter adjustment. A face detection and pose estimation algorithm isused to determine a general position and direction the face is pointingtoward, which aids in model position and alignment. Machine learningmethods are used to train a classifier for detecting a face as well asdetermining the pose of the head in an image that is post-processed todefine various features, including but not limited to Haar-Like or LocalBinary. Training datasets consists of images of faces in various posesthat are annotated with the location of the face and direction of pose,and also includes specific facial features. The output consists of alocation of the face in an image and a vector of the direction of headorientation, or pose.

Once the face and pose are established for the first image frame, aniterative process begins where more detailed facial features relevant toeyewear placement and general face geometry are defined, including butnot limited to eye location, nose location and shape, ear location, topof ear location, mouth corner location, chin location, face edges, etc.Again, machine learning is used to analyze the image to detect facialfeatures and edges. When these features are located, the generalizedquantitative anatomic model parameters is aligned and adjusted to findthe optimal fit with the features, minimizing the error between thedetected feature location and the mesh. Additional optimization of thegeneralized quantitative anatomic model may be performed to enhance thelocal refinement of the model using the texture information in theimage.

In an exemplary embodiment, the generalized quantitative anatomic modelhas parameters that influence features including but not limited to eyelocation, eye size, face width, cheekbone structure, ear location, earsize, brow size, brow position, nose location, nose width and length andcurvature, feminine/masculine shapes, age, etc. An estimation of theerror between the detected features and model is used to quantifyconvergence of the optimization. Small changes between adjacent imagesin the dataset are also used to refine pose estimation and alignment ofthe model with the image data. The process iterates to subsequent imageframes.

In an exemplary embodiment, features detected from adjacent image framesare used to initialize subsequent or previous frames to enhance featuredetection. The process continues through as many images as needed andpossibly cycle through images multiple times to converge on the optimalparameters to minimize error between the distorted generalized model andthe image data. Regularization and smoothing may be employed to minimizenoise and variance of features points, pose, and the anatomic modelfitting between frames. The final quantitative anatomic model will bescaled based on the reference data such as input from the user orscaling to a reference object as previously described. Alternatively, ifthe anatomic model was derived as a shape model in real-worlddimensions, the association between the shape and size of the face maybe used to directly provide the scale of the model.

Since the model was refined through a series of images, the orientationand geometric relationship between the model and image data is known. Abundle adjustment of the features points and face model across theimages may be performed, which provides precise camera locations thatregister the anatomic model to the image data. This information can beused to orient and register the model to the image data for subsequentrendering.

Those skilled in the art will recognize there are many ways to constructand represent quantitative information from a set of image data. Inanother embodiment, no prior generalized anatomy model is required togenerate a quantitative anatomic model. A method such as structure frommotion (SFM) photogrammetry is used to directly build a quantitativeanatomic model. In this technique, a series of images is required aroundthe user's face. The features detected in each image, and the relativedistances between the features from image-to-image are used to constructa 3D representation. A method that combines a generalized shape modelwith subsequent local SFM refinement may be utilized to enhance localdetail of features, such as the nose shape.

In another embodiment, the quantitative anatomic model consists only ofa point cloud of key features that are detected. For example, the centerof the eyes, corners of the eyes, tip of the nose, top of the ears, andother important landmarks is detected and tracked through multipleimages. These simple points, oriented in space in a dataset, provide allthe information needed to obtain quantitative information needed forsubsequent analyses. They may be obtained using the methods previouslymentioned, or with other methods like active appearance models or activeshape models.

Technologies such as depth cameras or laser sensors may be used toacquire the image data, and there exists prior art describing how thesetechnologies can directly produce 3D models, essentially like a 3Dscanner, by their ability to detect distance. Additionally, the use ofout of focus areas or the parallax between adjacent images is used toestimate depth.

Alternatively, the quantitative anatomic model and dimensions can bederived from a pre-existing model of the user's face that they possess.Models may be acquired from 3D scanning systems or imaging devices. If auser already has an anatomic model their face, they may digitallytransfer it to the computer system by non-transitory computer readablemedia, a network connection, or other means.

During acquisition of user image data for customizing products, such aseyewear, the scale and dimensions of the user are important to ensurethat the size of the resulting product is appropriate and that the userreceives a product that matches the previewed version. The followingembodiments describe various systems and methods for acquiring, scaling,and reconstructing anatomic models from image data:

Embodiment to Scale an Anatomic Model of a User's Face with a ReferenceTarget Present in Multiple Images

Referring now to FIG. 17, as to this embodiment, a) A computer system1701 is configured with a camera or imaging device 1702 used to acquireimage data of a user 1703; b) A reference target 1704 of knowndimensions (eg coin, credit card, phone, tablet, screen, paper, ruler,etc.) is positioned such that it is visible in at least some images ofthe user; c) The reference target has at least one predetermineddimension 1705 (eg, diameter of a coin); d) The computer systemreconstructs an anatomic model of the user's face based on the imagedata; e) The computer system detects the reference target in at leastsome images, including detection of the at least one predetermineddimension; f) The computer system co-registers the anatomic model withthe original user images such that the model coordinates and cameraposition align the face model with the pose, position, and scale of theimages of the user's face 1703; g) The computer system uses the ratio ofthe detected target dimension(s) and the known dimensions of thereference target in each image to set a scaling factor to the dimensionsof the anatomic model and h) The computer system may additionallyaverage or weight the measured dimensions of multiple predetermineddimensions of the reference target(s) in each frame in order to reduceerror from any single dimensional measurement.

Embodiment to Scale an Anatomic Model of a User's Face with a ReferenceTarget Present in Only One Image

In this embodiment, a) A computer system configured with a camera orimaging device is used to acquire image data of a user; b) A computersystem configured with a camera or imaging device is used to acquire aseparate image of a user with a reference target of known dimensionspresent in the image; c) The reference target has at least onepredetermined dimension (eg, diameter of a coin); d) The computer systemreconstructs an anatomic model of the user's face based on the imagedata; e) The computer system co-registers the anatomic model with theuser's image containing a reference target such that the modelcoordinates and camera position align the face model with the pose,position, and scale of the image of the user's face; and f) The computersystem uses the ratio of the detected target dimension and the knowndimensions of the reference target in the image to set a scaling factorto the dimensions of the face model.

Embodiment to Scale Image Data That an Anatomic Model of a User's Faceis Constructed From

In this embodiment, a) A computer system configured with a camera orimaging device is used to acquire image data of a user; b) A referencetarget of known dimensions (eg coin, credit card, phone, tablet, screen,paper, ruler, etc.) is positioned such that it is visible in at leastsome images of the user; c) The reference target has at least onepredetermined dimension (eg, diameter of a coin); d) The computer systemdetects the reference target in at least one image, including detectionof the at least one predetermined dimension; e) The computer system usesthe ratio of the detected dimension and the predetermined size of theobject to set a scaling factor to image data (e.g. to apply dimensionsto the size of pixels); and, f) The computer system reconstructs aanatomic model of the user's face based on the image data, with themodel assuming the underlying dimensions of the images

Embodiment to Scale an Anatomic Model of a User's Face with a ReferenceTarget Included in the Model

An advantage of this embodiment is that the orientation and position ofthe reference target with respect to the user's face is not as importantsince it will be reconstructed with a model.

Referring to FIG. 19, in this embodiment a) A computer system configuredwith a camera or imaging device is used to acquire image data of a user;b) A reference target of known dimensions (eg coin, credit card, phone,tablet, screen, paper, ruler, etc.) is positioned such that it isvisible in at least some images of the user; c) The reference target hasat least one predetermined dimension (eg, diameter of a coin); d) Asshown in FIG. 19, the computer system reconstructs a model (or models)of the user's face 1901 and the reference target 1902 based on the imagedata where the face and target may or may not be in contact with eachother, so there are two models positioned in space relative to oneanother; e) The computer system detects the reference target in themodel, including detection of at least one predetermined dimension; f)The computer system uses the ratio of the detected dimension of thereference target in the model and the predetermined size of the targetto set a scaling factor to overall model; and g) Optionally, thecomputer system removes the reference target from the model afterscaling, leaving only the final scaled face model.

Embodiment to Scale an Anatomic Model of a User's Face with PupillaryDistance (Pd) Input by a User

In this embodiment, users commonly have Pd measured by theiroptometrist, which provides a reference dimension to scale the headwith. How this is done is as follows: a) A computer system configuredwith a camera or imaging device is used to acquire image data of a user;b) The computer system reconstructs an anatomic model of the user's facebased on the image data; c) The computer system detects eye features ofthe user (pupils, irises, etc) in the face model and measure thedistance between the eye features; d) Before, after, or during the imageacquisition and reconstruction process, the user provides their Pdmeasurement; and, e) The computer system uses the users Pd measurementto set a scaling factor to the dimensions of the model, adjusting thesize of the model such that the measured eye distance in the modelmatches the user's actual Pd.

Embodiment to Scale an Anatomic Model of a User's Face with DimensionsDetected and Measured in Image(s) and Then Applied to Scale a Model ofthe User's Face

In this embodiment, a) A computer system configured with a camera orimaging device is used to acquire image data of a user; b) A referencetarget of known dimensions (eg coin, credit card, phone, tablet, screen,paper, ruler, etc.) is positioned such that it is visible in at leastsome images of the user; c) The reference target is determined to haveat least one predetermined dimension (eg, diameter of a coin); d) Thecomputer system detects the reference target in at least one image,including detection of the at least one predetermined dimension; e) Thecomputer system detects facial features (pupils, irises, eye corners,mouth corners, nose, etc) in at least one image and measure theun-scaled distance between them; f) The computer system reconstructs ananatomic model of the user's face based on the image data; g) Thecomputer system uses the ratio of the detected dimension of thereference target in the images and the predetermined size of the targetto set a scaling factor to the detected facial features (Pd, distancebetween eye corners, width of mouth, etc); h) The computer systemdetects the facial features in the face model, measures the distancebetween them, and uses the scaled facial feature measurement to scalethe face model; and, i) Optionally, the computer system detects thefacial feature directly in a face model registered to the image datawithout first detecting the facial features in the image data.

Embodiment to Scale an Anatomic Model of a User's Face by DeterminingDepth with a Reference Target Present

In this embodiment, a) A computer system configured with a camera orimaging device is used to acquire image data of a user; b) A referencetarget of known dimensions (eg coin, credit card, phone, tablet, screen,paper, ruler, etc.) is positioned such that it is visible in at leastsome images of the user; c) The reference target has at least onepredetermined dimension (eg, diameter of a coin); d) The computer systemdetects the reference target in at least some images, includingdetection of the at least one predetermined dimension; e) as shown inFIG. 17 the computer system 1701 uses the detected dimensions 1705, theknown size of the reference target 1704, and intrinsic camera parametersto determine the distance 1706 from the camera to the target; f) Thecomputer system reconstructs a model of the user's face based on theimages; g) The computer system uses the distance to the reference targetand user's face and intrinsic camera parameters to determine the scaleof the user's face model; and, h) Optionally, the computer systemaverages the measured dimension of the reference target from multipleframes to reduce error from any single image measurement prior toscaling the face model.

Embodiment to Scale an Anatomic Model of a User's Face Using a ComputerSystem with Depth Detected in Images

In this embodiment a) A computer system configured with a camera orimaging device with depth sensing capability is used to acquire imagedata of a user; b) The user positions the computer system to obtainimages of themselves, while the computer system also measures distancefrom the computer to the user (rangefinder, autofocus distance, depthsensor, etc); c) The computer system uses the distance measured from thecomputer to the user and the intrinsic camera parameters to determinethe scale of the images; and, d) The computer system reconstructs amodel of the user's face based on the image data; with the model beinginherently scaled based on the dimensions in the images.

Embodiment to Scale an Anatomic Model of a User's Face Using a ComputerSystem with Depth Detected at Each Pixel

In this embodiment a) A computer system configured with a camera orimaging device with depth sensing capability is used to acquire imagedata of a user; b) The user positions the computer system to obtainimages of themselves, while the computer system also measuring distancefrom the computer to each pixel in the image data; c) The computersystem uses the distance measured from the computer to the user at eachpixel and uses the camera intrinsic parameters to scale each pixel ofthe image data; and, d) The computer system reconstructs a model of theuser's face based on the image data, applying the scale of each pixel tothe model, such that the model is scaled when completed.

Embodiment to Scale an Anatomic Model of a User's Face Using a ComputerSystem with Depth Detected Only at Close Distances

In this embodiment a) A computer system configured with a camera orimaging device with depth sensing capability is used to acquire imagedata of a user; b) A computer system configured with a camera with depthsensing capability is used to acquire close-up image data of a user, forexample, including at least the user's eyes or other facial features inthe image data; c) During acquisition of the close-up image, the userpositions the computer system to obtain an image of at least some facialfeatures, while the computer system also measures distance from thecomputer to the user; d) The computer system detects facial features(iris, pupil, etc) in the close-up image and measure the distancebetween the features; e)The computer system uses the distance measuredfrom the computer to the user and intrinsic camera properties todetermine the scale of pixels in the image data; f) The computer systemdetermines reference distances between facial features based on theimage scale and measured distance between features; g) The computersystem reconstructs a model of the user's face based on the image dataof the whole face of the user; and, h) The computer system detectsfacial features in the face model, measures the distance between them,and uses the reference feature measurement to scale the face model.

Embodiment to Scale an Anatomic Model of a User's Face Using a ComputerSystem and a Double Mirror Reflection

Referring to FIG. 20, in this embodiment a) A computer system 2001configured with an imaging device 2003 and a display 2008 on the sameside as the imaging device is used to acquire image data of a user 2004;b) The user 2004 acquires images in front of a mirror 2007 with thedisplay 2008 and imaging device 2003 facing the mirror 2007 so theysimultaneously acquire image data of the user and the device displayingpreviews of the image data which is also captured by the imaging devicethrough mirror reflection; c) The computer system detects at least onedimension of the computer system in the image (size of screen, size offeature on computer, reference image on computer, etc); d) The computersystem determines the known reference size of the detected dimension byproviding its make/model, screen dimensions, size of reference image,etc.; e) The computer system detects at least one dimension (distancebetween eye features, size of head, model dimensions, etc) in each ofthe simultaneous sets of image data of the user (the user and the useron the display of the device); f) The computer system 2001 uses thereference dimension of the computer system and the intrinsic cameraproperties to determine the distance 2009 between the device and themirror; g) The computer system uses the distance between the device andthe mirror, the detected user dimension on the display of the device,the detected user dimension in the mirror, and the properties of theimaging device to set a scaling factor of the detected user dimensions;h) The computer system reconstructs a model of the user's face based onthe image data; i) The computer system detects the user dimension(s) onthe reconstructed model and scales the model based on the scalingfactor; and j) Optionally, the user may place or hold a reference objectagainst the mirror to determine the distance from the computer system tothe mirror.

Embodiment to Scale an Anatomic Model of a User's Face Using Front andRear Cameras of a Computing Device

Referring again to FIG. 20 in this embodiment a) A computer system 2001configured with a imaging devices on the front 2002 and back 2003 of thecomputer system a is used to acquire image data of a user; b) The user2004 acquires image data in front of a mirror 2007 so theysimultaneously acquire image data of the user with one camera (direction2005) and an image of the reflection of the user with the oppositecamera (direction 2006); c) The computer system detects at least onedimension of the computer system in the image data (size of screen, sizeof feature on computer, reference image on computer, etc); d) Thecomputer system determines the known reference size of the detecteddimension by providing its make/model, screen dimensions, size ofreference image, etc; e) The computer system reconstructs an anatomicmodel of the user's face based on the image data with the computersystem optionally using the pair of image data together as stereo datato enhance 3D reconstruction; f) The computer system registers theanatomic model on both sets of image data; g) The computer system usesthe reference dimension, the registered anatomic models, and camerainstrinsic parameters to determine the scaling factor of the model; and,h) Optionally, the user places or holds a reference object against themirror to determine the distance from the computer system to the mirror.

Embodiment to Scale an Anatomic Model of a User's Face Using a ComputerSystem and a Mirror

In this embodiment a) A computer system configured with a camera orimaging device is used to acquire image data of a user positioned infront of a mirror with the camera positioned near their face; b) Areference target of known dimensions (eg coin, credit card, phone,tablet, screen, paper, ruler, etc.) is positioned such that it is on thesurface of the mirror and visible in at least some images of the user;c) The reference target has at least one predetermined dimension (eg,diameter of a coin); d) The computer system detects the reference targetin at least one image, including detection of the at least onepredetermined dimension; e) The computer system reconstructs an anatomicmodel of the user's face based on the image data; f) The computer systemuses the camera instrinsic parameters, the detected reference dimension,and the known dimensions of the reference object to determine thedistance from the camera to the mirror. Since the mirror is the midpointbetween the user and the reflection of the user seen by the camera, thedistance from the camera to the user is 2× the distance from the camerato the mirror; g) The computer system uses the distance from the camerato the \user and the camera intrinsic parameters to set the scale of theimage data; and. h) The computer system reconstructs an anatomic modelof the user's face based on the image data.

Embodiment to Build and Scale an Anatomic Model of a User's Face with aCollection of Previously Acquired Images

This embodiment has the advantage of using a collection of previouslyacquired images that the user may have at their disposal (eg collectionof existing photos, photo gallery, social network or online imagegallery photos, etc). Referring to FIG. 21, in this embodiment, a) Acomputer system receives a collection of images (e.g. 2101, 2102, 2103)of a user 2105, b) The images may be previously tagged with facialrecognition data to determine which face in each photo is the user. c)If the images were not previously tagged, then the computer systemperforms facial recognition, prompting the user to confirm which face istheirs in at least one image or using the highest frequency of detectedfaces to determine the user from other people in photos, d) The computersystem detects facial features (eg various points of the eyes, nose,mouth, ears, chin, etc) in each image of the user and fits a face model2104 to the image data, e) Optionally, the computer system determinesexpression in each image (e.g. 2101 vs 2103) and adjusts the face modelto a neutral expression, f) The computer system determines the pose ofthe user's face in each image, g) The computer system reconstructs asingle model 2104 of the user's face by fitting a face model across thecollection of feature sets and camera positions (2105, 2106, 2107) ofthe user. The face model is scaled by one of these methods: h) Thecomputer system requests additional data from the user based onpreviously described methods: Pd input, an image with the referencetarget, etc. i) The computer system detects known objects in the imagesto determine a reference size (eg recognize a sheet of paper, a logo, aphone, etc). j) The computer system requests additional image data takenof the user with a reference object, using any other method describedherein. k) The face model is inherently scaled due to the shape modelcontaining dimensions that relate shape and size.

Embodiment to Scale a User's Face Using Existing Eyewear That TheyAlready Possess

Many people shopping for eyewear already own eyewear, and whether theeyewear fits well or not, it is used to help scale the dimensions of theuser's face. Alternative, the manufacturer could send a sample pair ofeyewear to be used for this process.

Referring to FIG. 22, in this embodiment, a) A computer systemconfigured with a camera or imaging device is used to acquire image data2201 of a user 2202, b) A computer system is used to acquire separateimage data 2203 of a user wearing eyewear 2204 they possess, c) Thecomputer system requests that the user provide reference dimensionalinformation about the eyewear, such as the width 2205 or length of theframe, the size of the lenses, etc (e.g. a photo of the eyewear next toa reference target used to scale the eyewear, a measurement of theeyewear 2207 by aligning it with a reference 2208 on the computer systemdisplay 2206 that is set to 1:1 scale as explained in later embodiments,an entry of a measurement, model name of the eyewear, a ruler orinteractive ruler displayed on the screen that the user can utilize tomeasure their eyewear, etc), d) The computer system reconstructs a modelof the user's face based on the image data, e) The computer systemdetects the eyewear dimensions in the image data (eg overall width orheight of frame, width of lens, etc), f) The computer system associatesfeatures or a model of the user's face between the image data with andwithout eyewear (e.g. eye 2209 and mouth corner 2210), g) The computersystem determines a scaling factor for the face model based on thedetected and reference eyewear dimensions and the features associatedbetween the image data with and without eyewear, and h) The computersystem co-registers the face model with the original user images suchthat the model coordinates and camera position align the face model withthe pose, position, and scale of the images of the user's face.

Embodiment to Scale a User's Face Using Sonar

For any embodiment that requires calculating a distance from thecomputer system to the user or computer system to a mirror, a sonarmethod is used.

The following embodiment describes using sound to determine distance. a)A computer system configured with a camera or imaging device is used toacquire image data of a user, b) A computer system further configuredwith a microphone and speaker is used to emit a sound (e.g. series offrequencies, repeated sounds, etc) and record the same sound with amicrophone, c) The sound is emitted from a on-the-device-speaker, aheadphone on the user or held a distance, or other device, d) A computersystem calculates the distance between itself and an object, such as thedistance from the computer system to a mirror or distance from aheadphone in the user's ear and the computer system, by analyzing thetime elapsed from the sound being emitted by the computer system tobeing detected by the computer system's microphone, e) The computersystem may use use multiple sounds, filtering, or other analysis toreduce noise, reflections, artifacts, and to optimize the accuracy ofthe distance detection, and f) The computer system uses the distance asdescribed in other embodiments for scaling image data or an anatomicmodel of the user.

Embodiment to Determine Pd from Face Model that is Already Reconstructedand Scaled

In this embodiment, a) A computer system obtains a scaled face model ofa user from image data (using any method previously described), b) Thecomputer system detects features of the eyes from the face model(irises, pupils, etc), and c) The computer system measures the distancebetween the eye features on the face model to calculate Pd

Embodiment to Provide Users a Means to Measure the Size of a ReferenceObject of Their Choice

For any embodiment requiring a reference object of a known dimension,there are situations where the user needs to use an object that they orthe computer system do not know the dimension of, ie a business card, apencil, eyewear they possess, etc.

This particular embodiment describes a system to measure rectangularobjects (or objects that can fit within a rectangle) of unknowndimensions, but the method could be extended to any shape. Referring toFIG. 23: a) A computer system 2301 configured with a display 2302 andinput device is used to display a reference box 2303 on the display, b)A computer system obtains information about the display of the computersystem, such as resolution, pixel size, overall display dimensions. Thecomputer system obtains this information from itself, software on thecomputer system, from a web browser, from the user providing informationabout the display or computer system model, c) A computer systemcalculates the pixel size of the display (for example, by dividing thelength and width of the screen by the number of pixels). d) The computersystem then calculates the true size of the reference box 2303 on thedisplay, e) The computer system instructs the user to place theirreference object 2306 against the screen and adjust as illustrated at2305 the reference box 2303 using an input device (touchscreen, mouse,touchpad, gesture, etc) to match the size 2307 of the object, f) Thecomputer obtains the size of the reference object by calculating thesize of the adjusted reference box, and g) Optionally, the computersystem is configured with an imaging device 2308 to take image data ofthe reference object such that it obtains information about theappearance of the object for recognition in future images. If thecomputer system is configured with a depth image device, it uses depthand scale information to enhance the measurement of the referenceobject.

For any embodiments that involve using a reference object, the objectdoes not need to be perpendicular to the imaging device to obtain properdimensions. With prior knowledge of the reference object, the angle ofthe object relative to the camera is determined. The angle and themeasured distance on the image plane is used to determine to the truereference dimension of the object.

Optional User Preferences and Information

FIGS. 1A and 1B at step 104 describe capturing the user's prescriptiondata and other information to inform the analysis. This step may beperformed at a later time, although there is an advantage to capturingthe data while the computer system analyzes the image data if it iscomputationally time-consuming. The computer system requests thisinformation through a form that the user enters information into bymeans of an input device connected to the computer system. The computersystem may also receive the information by obtaining image data of aphysical set of information, such a photo of a prescription. Thecomputer system may use optical character recognition to decode theimage and extract the user's prescription data. The computer system mayreceive the user information though voice recognition, electronicallytransferred data or other means. The use of the information entered bythe user will be described later in a description of modeling lenses andcreating custom eyewear models.

Configurable Product Model

In FIGS. 1A and 1B steps 106 and 107 describe a configurable product orconfigurable eyewear model. In an exemplary embodiment, the configurablemodel is three-dimensional, configured with parametric features anddimensions, and represented as a 3D surface mesh. A 3D model of eyewearis created from a variety of methods such as 3D capture via scanning orphotogrammetry, or through 3D computer aided drafting (CAD) or 3Dmodeling. It should be noted that a variety of other methods orrepresentations of a configurable model could be used, such as 2Dmodels, shape models, feature-based models, etc.

In an exemplary embodiment, a 3D parametric model is created by theeyewear manufacturer, including the frames and or frames and lenses. The3D parametric model is created as a surface mesh or a solid model madeof elements or features, including but not limited to polygons,curvilinear elements, etc. The parametric model enables altering one ormore dimensions of the eyewear, which would update appropriate model andmesh elements, while maintaining consistent relationships between otherfeatures.

FIG. 9 shows an example of an eyewear model 901 that was adjusted toeyewear model 902 by altering a parameter for the width 903 of theeyewear around the lens. The advantage of the parameterized eyewearmodel is that the width 907 of the bridge and nose pads is retained, theheight 908 is retained, and the overall aesthetic appearance betweeneyewear models 901 and 902 is consistent. The parameterization enables asubstantial change to just one aspect of the frame 901 without affectingother important elements of the design. The parameterized eyewear modelhas an advantage in propagating changes from a feature to the rest ofthe model while constraining all other features. These changes arerepresented as simple numeric values, which allows for very efficientdata transfer and storage. These parameters could have up to infinitevariability of the size and form of the product, allowing ultimateprecision, if needed, in fitting a custom model to a user's anatomy andpreferences. The ability to have high or infinitely variability of theform of the glasses in this example demonstrates a fundamental principleof one-up, from-scratch custom products. By changing and customizing theunderlying form of a major component, in this case the front frame, in ahighly unique manner that could never be done with pre-manufactured orstock components, the design is inherently one-up and custom-made forthe individual user.

FIG. 13 illustrates an example base eyewear design 1301, whichdemonstrates further shape customization. A base design is a fundamentalstyle or shape that the eyewear model has, which may be modified throughconfiguration and parameters. A computer system adjusts the curvaturebetween points 1305 and 1307. Or a user directing a computer systeminput device selects a point on the eyewear at 1305 and move along thedotted line in the direction of the arrow 1306 to point 1307. Theeyewear 1302 would then be modified in the region 1308 that was edited.To retain symmetry while simultaneously reducing the number of stepsnecessary to customize eyewear, a change on one side of the eyewear isequally applied to the other side of the eyewear, as shown in updatedeyewear 1303. This symmetry effect is one example of a constraint thatmay be introduced as a feature of a configurable model.

The configurable eyewear model has constraints that prevent certain keyparts/regions from being altered into a design that is no longer optimalto manufacture. For example, the minimum thickness of parts is limitedto ensure structural strength, the minimum thickness around the lensesis limited to ensure the lenses can be assembled into the eyewearwithout the eyewear breaking, and the possible hinge locations islimited to ensure they could fit and sit at a proper angle. If aparticular stock component hinge must be used, then the connection pointof the hinge must be consistent regardless of how the underlying formand shape of the custom eyewear changes. Additionally, certain featuresare related due to symmetry or cascading effects; for example, if thecomputer or user adjusted the width or thickness of one part of the rim,the entire rim on both sides would adjust to ensure a symmetric andattractive appearance. The overall location of features remainconstrained, such as the hinge and nose pad locations, etc. All theseconstraints and relationships would be pre-programmed by the eyeweardesigner and would be incorporated in the configurable model.

FIG. 29 illustrates an example of customization achieved withconfigurable product model; in particular, the ability to combinevarious parameters to refine and customize a product model. An eyewearmodel 2900 is configured to the 16 variations in the illustration. The 4columns 2902 illustrate example configurations of the eyewear lens width2903 and height 2904. The 4 rows 2901 illustrate the combinations ofvarying parameters for nose bridge width 2905, the distance 2906 betweenthe temples where they contact the ears, the height 2907 from the frontframe to the ears, and other subtle changes. Key features such as thematerial thickness 2908 and the hinge size and location 2909 remainunchanged. The parametric configuration enables the eyewear design to behighly configurable while remaining manufacturable. A manufacturer mayuse 1 hinge and 1 material thickness for all these designs and more, yetstill allow massive customization of the underlying shape and size.Models 2900 and 2910 are quite distinct and they would traditionalrequire different mass produced products. It would be completelyimpractical to offer this level of variation to customers withtraditional mass-produced products, requiring thousands, millions, ormore components to be designs and stocked. A configurable model with therest of the method and system described herein allows one base model tobe configured in all the configurations illustrated in FIG. 29, so oneproduct can be custom tailored to an individual customer and thenproduced. It should be noted that these 16 variations represent anextremely small subset of the total potential variation of the design;there are thousands, millions, or infinite variation possible byinterpolating between the examples shown, extrapolating beyond, andconfiguring other parameters not shown in the illustration. For example,if a configurable model has 10 parameters that can be altered; eachparameter has 20 increments (which could also be infinite) such asdistances of 2 mm, 4 mm, 6mm, and so on; and the model is available in20 colors and 3 finishes; then the total combinations of configurationsfor that one model would be 6×10²¹, or six sextillion, which is 6000multiplied by 1 billion multiplied by 1 billion. It should also be notedthat these types of configurations are not limited the type that consistof replacing and combining off-the-shelf parts. The fundamental shapeand size of the components are entirely different for each parameterthat is changed, requiring a model that is configurable and that theparts are made from scratch. This degree of customization can only beachieved with one-up, from-scratch custom methods described herein.

In addition to geometry, the eyewear model may have parameters for thesurface finish, color, texture, and other cosmetic properties. The 3Deyewear model may be texture mapped with an image to represent thesurface or rendered with texture, lighting, and surface properties suchas reflectance, transmission, sub-surface scattering, surface orroughness to represent photo-realistic appearance of eyewear. Theconfigurable nature of the model would allow a multitude of materials,paints, colors, and surface finishes to be represented. Variousrendering techniques known to those skilled in the art, such as raytracing, are used to render the eyewear and lenses in the mostphotorealistic manner possible, with the intension to accuratelyrepresent and reproduce on the display the frame and lenses exactly ashow they would appear when manufactured. Other optical interactioneffects, such as shadows and reflections, can be displayed on theeyewear and on the 3D model of the user's face. The 3D eyewear model hashinge points at the temples to allow the temples to flex with respect tothe frame front and fit to the user's face model. In another embodiment,the 3D eyewear model also allows for a suitable amount of elasticmodulus (stretch) in the bulk material property of the frame, and thiselastic property can be dependent on the frame material selected.

Product Customization

Once an anatomic model is constructed, it is used to inform theplacement and customization of a configurable product model. In anexemplary embodiment, the computer system automatically adjusts and theeyewear to the user's face based on at least one of: the quantitativeanatomic model, the user's preference inputs, and the user's image data.The dimensions of the quantitative anatomic model and configurableeyewear model are both known to the computer system, various sizeadjustments are made automatically to ensure the best fit or arrive to asolution that is very close to the best fit. Three different approachesare described: a method to customize the configurable eyewear modelprior to alignment/placement with respect to the anatomic model andrendering previews for the user, after alignment/placement with respectto the anatomic model but before rendering previews for the user, andafter alignment/placement and rendering previews for the user, such thatthe user can provide additional input after seeing the basepre-configured eyewear model on their face.

Customization Prior to Placement on Anatomic Model

In one embodiment, the eyewear model is automatically customized priorto being positioned on the anatomic model; therefore creating anentirely new and custom design before ever fitting or rendering itdirectly to the user's images:

Refer to FIG. 30. In this embodiment, a) A computer system obtains ascaled face model 3001 (using any previously described method) that haskey facial features 3005 identified, including but not limited todimensions, points, lines, and surfaces of the eyes, nose, ears, brow,etc., b) The computer system obtains a configurable 3D product model3002 that has key features 3006 identified, including but not limited todimensions, points, lines, and surfaces of the temples, nose pads,lenses, bridge, etc. c) The computer system performs an optimization ofthe configuration product model parameters to reduce the error betweenvarious features of the face and model based on predefined fit metrics,such as the optimal ratio of eyewear width to face width, the optimalcentering of eyes within lenses, etc. For example, adjust the length ofthe temples until the error between the temples and top of the ear areminimized. Or the computer system optimizes the fit and style based onother techniques, such as machine learning or analytic equations. d) Thecomputer system updates the configurable product model 3003 with newparameters. e) The computer system performs an optimization to obtain arigid transformation, as illustrated at 3004, to align the product model3003 to the face 3001. The error between key features of the product andface is minimized, and some features are weighted more than others. f)The computer system transforms the coordinates of the product model toalign it with the anatomic model, thereby placing a new eyewear designaligned with the user's anatomy.

Customization After Placement on Anatomic Model

In another embodiment, the base eyewear is positioned relative to theanatomic model and then automatic adjustments are completed as follows,creating an entirely new custom product prior to rendering for theuser's preview. Refer to FIG. 31

a) A computer system obtains a scaled face model 3101 (using anypreviously described method) that has key facial features 3107identified, including but not limited to dimensions, points, lines, andsurfaces of the eyes, nose, ears, brow, etc. b) The computer systemobtains a configurable product model 3102 that has key features 3108identified, including but not limited to dimensions, points, lines, andsurfaces of the temples, nose pads, lenses, bridge, etc., c) Thecomputer system performs an optimization to obtain a rigidtransformation to align the product model to the face, as illustrated at3103. The error between key features of the product and face isminimized, and some features are weighted more than others. d) Thecomputer system transforms the coordinates of the product model to alignit with the anatomic model. As illustrated at 3104, the computer systemanalyzes the interactions and dimensions and errors between the productmodel and anatomic model. In the example illustration, the eyewear modelat 3103 is too large for the user's face, sits too low due to the nosesize, and is too wide for the face shape. e) The computer system thenautomatically adapts the product model as illustrated in 3105 to furtherminimize errors between the facial features and product features basedon predefined fit metrics, such as the optimal ratio of eyewear width toface width, the optimal centering of eyes within lenses, etc. Theresulting custom model 3106 is better designed for the user.

Custom Fitting

The computer analyzes a set of measurements between the quantitativeanatomic model and eyewear model. The set of measurements include but isnot limited to: Width of eyewear relative to width of face; Distancebetween nose pads relative to width of nose; Angle, shape, or size ofnose pads relative to angle, shape or size of nose; Length of templesrelative to ear position; Height of eyewear relative to height of face;Height of each ear with respect to the eyes or other reference points;Distance between lens centers and eye centers; Vertex distance frominside lens surface to pupil; Outward angle of temples relative toframe; of lenses relative to the plane created by the front of the face;Eyewear wrap angle vs corresponding wrap curvature of the face.

The computer system uses these measurements to optimize a configurableeyewear model to the user's face. The automatic adjustment is informedby default metrics, such as optimal values for a ratio ofeyewear-to-face width. Ideally each metric is a dimensionless ratio thatscales properly across all user faces. However, some measurements, suchas vertex distance may be specified dimensions. Ranges of optimal valuesmay be used as well. Each metric is optimized individually, or they areoptimized together if there is an interaction effect, such as theinteraction effect between eyewear frame width and temple angle.

For example, FIG. 14 shows a user quantitative anatomic model 1401 andconfigurable eyewear model 1402 in view 1411 before automaticoptimization. A set of metrics is the ratio of width of eyewear 1403 towidth of face 1404, angle of temples 1407, and length of the entiretemples 1406 relative to the distance to the top of the ear 1405. Asonly one example, the optimal values for these metrics are 0.95, 87degrees, and 1, for which the pre-optimized eyewear model 1402 does notsatisfy. The computer system would seek to minimize the error betweenall three metrics and the optimal values. An optimization method such asleast-squares, steepest descent, or others known to those familiar withthe art is used to obtain a new set of eyewear parameters that best fitthe users face. After the parameters are updated, the automaticallyadjusted 3D eyewear model as shown in 1412 is displayed, enabling abetter first-visualization or approximation of all the eyewear models,as the width 1408, temple length 1409 and temple angle 1410 are bettersuited for the user. Automatically sizing the eyewear to a best-fit orclose to best-fit size for the user enables a better shopping experiencedue to the reduced time and steps the user must take to arrive at thefinal eyewear design. The user may also be delightfully surprised seeingthemselves in a pleasing eyewear design they had not preconceived or instyles they did not previously know would suite them. The concept ofmaking every design and style fit well is a great first step to ensuringa good shopping experience.

By way of another examples, FIG. 28 illustrates a cross section of anose 2801 and eyewear model 2802 prior to customization. The nose pads2803 do not match the contour of the nose and intersect with the surfaceof the nose. The same nose 2804 is illustrated with eyewear model 2805,which was custom configured for the user. The nose pads 2806 now matchthe contour and angle of the nose and sit nicely on the surface. This isan example of the exceptional power of full customization, as the priorstate-of-the-art does not enable full customization of nose pad contoursto precisely match and fit a user's nose.

In some cases, when the eyewear model is highly configurable or theoptimal values are well within the solution space of the parameterizeddesign, no optimization is needed and a direct solution of the exactlyspecified metrics can be obtained. For example, if the temple lengthneeds to be 103.4 mm and the front width of the glasses needs to be142.1 mm, then the model could be adjusted to exactly these values.

Optimal values may vary based other factors entered by the user ordetermined from the image data, such as gender, age, face shape, eyewearstyle, purpose of eyewear, or what is currently fashionable. Forexample, females may prefer on average slightly smaller eyewear relativeto their face size than males. Users choosing eyewear for recreationaluse may prefer increased frame wrap and a tighter temple fit to reducewind in their eyes, widen their field of corrected vision, and/orprovide more impact or sun protection. Users choosing plastic eyewearmay prefer larger eyewear than users choosing metal eyewear. Theseuser-defined preferences may be used to alter the optimal parametersduring the customization process.

Customization and Aesthetic Prediction

In an exemplary embodiment, the custom fit and style is recommended bythe computer system based on the user's image data and potentiallyadditional information they provide. In addition to custom fit for abase design selected by the user, the computer system may also suggesteyewear styles, creating custom products specific for the user.Information about the user obtained from his imaging data and anatomicmodel that is used to provide custom suggestions includes but is notlimited to:

Overall face size, such as area of the front of the face or volume ofthe head from the model; Face width; Face height; Ear positions (eachear may have different heights); Inter- pupillary distance; Size ofeyes, such as area or length or height; Spacing between eyes;Asymmetries of the nose, eyes or mouth; Color of eyes; Color of hair;Amount and shape of hair; Color of skin; Ethnicity; Age; Location orlocal style trends; Gender; Assessment of cheekbone shape and location;Angle of forehead; Angle of cheeks; Circles under eyes; Eyebrow size andshape; Shape of face (e.g. round, square, oval, etc); Vertical positionof eyes relative to center of face; Hair style (e.g. up, down, long,balding, straight, curvy); facial hair; Intensity or softness offeatures.

A portion of, all, or additional features are defined from the imagedata. Some features are directly measureable on the quantitativeanatomic model. For example, the curvature of the nose and the positionsof the ears are directly measureable from the anatomic model. In anexemplary embodiment, machine-learning algorithms are used to classifyfeatures. A training database of image data from a plurality of faces iscollected and all features recorded. The computer system performs aplurality of analyses on each image, such as intensity maps, gradientfilters, Haar filters, hessians, Sobel filters, Hough transforms,segmentation, and canny filters in order to measure or detect aplurality of features such as mouth angles, face edges, nose sizes,wrinkles, etc. For example, to estimate a wrinkle feature to aid withestimating age, the computer system analyzes the portion of the imagesegmented by the anatomic model. Within the bounds of the model, a Sobelfilter is applied to detect edges and the intensity of edges. The faceregion is subdivided into a plurality of regions where the Sobel filteris applied, and the quantity and intensity of edges is quantified withineach region. The sum of all regions for the face provides a feature todetect wrinkles. A person without wrinkles, who will only have edgefeatures at key facial features such as eyes and mouth, will have acomparatively lower score than people with wrinkles, who will have moreedge features due to their wrinkles. A machine learning method is usedto classify the features in the training set, including but not limitedto support vector machine, boosting, bagging, random forests, etc. Thecomputer system then uses the machine learning classifier to relateimage data to the desired features.

Other aesthetic characteristics may be quantified as well. Detectingskin features or hair features using previously mentioned techniquesallows for those regions of the image data to be isolated. Imageanalysis of color would then allow a characterization of skin tone andhair color to be established. Clustering is a method that would enablecategories of skin tones or hair colors to be established, groupingsimilar colors from the image together. Alternatively, machine learningmethods could be used on the color space of the image data to train aclassifier for determining aesthetic characteristics.

In an exemplary embodiment, the user is asked to provide someinformation as well to enhance or supplement data that is analyzed fromhis image data. Users provide information including but not limited to:Age; Gender; Location; Occupation; Style preferences such as ‘trendy’ or‘traditional’; the type of outfits they would like to wear glasses with(formal, casual, etc); Color preferences; Their favorite clothing;Preferential rating of different eyewear styles or shapes; and Wordsthat describe themselves or their tastes.

Each feature may also carry a corresponding weight that signifies to thealgorithm the importance of said feature. Alternatively, a user may linka social network website, personal profile, advertising databaseinformation about the user, or other such source of personal informationto the computer system. This enables the computer system to import avariety of information about the user beyond what is practical to askthem, such as lists of their favorite music, celebrities, places theyhave visited, restaurants they like, or a language analysis of words anddescriptors they use publicly. For example, if a user's posts on a blogor social website are analyzed, it may become apparent that ‘red’ is acolor they mention far more frequently than other colors or that theywear dark formal clothing most frequently in their images, which couldbe used to inform the computer system about the user's color or stylepreference.

In an exemplary embodiment, the computer system would have a trainingdatabase of preferences associated with the various features. Thesepreferences include but are not limited to: Eyewear style, Eyewearmaterial, Eyewear shape, Eyewear color, Eyewear finish, Eyewear sizeincluding local size adjustments, including overall size and customlocal adjustments such as width, thickness, etc., Eyewear position onface, and Lens size.

The preferences are determined by actual users, designers, test users,or through other means. The preferences are set as a single favorite,plurality of favorites, range of favorites, ranked favorites, or scoredfavorites. Additionally, a user may have unfavorable preferences, orfeatures that do not appeal to them. For example, a user may equallyfavor round and oval frame shapes, but dislike rectangle frame shapes.The preferences are set automatically based on the user's use of thecomputer system. In an exemplary embodiment, the user shops for eyewearand when he takes certain actions, such as rating the eyewear, addingthe eyewear to their shopping cart, changing the eyewear, or answeringquestions about eyewear during their shopping process, the computersystem records and associates his actions with preferences. For example,if a user repeatedly tries on, likes, and alters eyewear to have bluecolors, then the color blue would be associated as a preference for thatuser.

In another embodiment, these preferences may be established with expertdesigners or test users. The designers or test users would progressthrough a specific set of questions or activities that require them torank or rate various eyewear designs and features. They may also beasked to modify or customize eyewear to their preference. Based on thedetailed testing of these users, a database of their preferences couldbe established.

The database then consists of a relationship between a plurality ofvariables: user's image data, quantitative anatomic models, and providedpersonal information; the analyzed data about users and their imagedata; and the preferences they have set. The computer system appliesmachine learning or predictive analysis to build a prediction of theresponse (preferences) based on the inputs from a new user: his newimage data and anatomic model, personal information, and shoppingbehavior on the computer system. This approach enables an advantage ofproviding a highly customized and convenient eyewear shoppingexperience. For example, a user's image data analysis and a few basicanswers to questions provides the following detailed profile of thatuser: a woman in mid-30s, dark medium-length hair, a square face, verysmall nose, slightly blue eyes, medium skin color, trendy fashion taste,white-collar profession, prefers bold fashion, wears glasses daily, andlives in an urban area. Each of these features may be associated withvarious eyewear preferences, and the combined information whenclassified by the machine learning method is able to recommend a set ofeyewear that truly matches the user's preferences, even if she has notstated or does not know a priori her eyewear design preferences. Whencoupled with the methods to automatically size the eyewear, in theeyewear shopping implementation described herein, the user starts hershopping experience with a highly personalized experience, and arrivesat a more ideal custom eyewear faster and easier than she would havethrough other existing shopping implementations.

In another embodiment, the product model is customized for asymmetries.For example, FIG. 24 shows a user 2401 with common issues of a crookednose 2402 and one ear lower than the other at 2403. These anatomicasymmetries of the face are present in many people and affect the wayeyewear fits or looks on their face, often requiring manual correctionby optometrists, which may or may not fix the problem. On user 2401, theeyewear 2404 sits at an angle 2405 and is shifted to the side due toasymmetrical facial features. In any previous embodiments forcustomization, the product model could be adapted differently for theleft and right side of the face. This could be achieved through havingdifferent measurements, points, surfaces, or other geometries tooptimize for the left and ride size of the product. The resultingeyewear may have different dimensions for the features on the left andright side, for example the temples are different lengths or the nosepad is shifted to one side. An additional constraint is added to theoptimization to achieve a horizontal and well-aligned placement of theeyewear, irrespective of the user's asymmetrical features. Afterasymmetrical customization, user 2401 has eyewear 2406 that sits leveland centered on the face.

It is desirable to design custom eyewear accounting for the user's facein various expressions. For example, the structure of cheeks change whena person smiles or brow shape change when a person frowns, which couldcause interference with the eyewear design, resulting in eyewear movingor being uncomfortable during normal use. The following embodimentdescribes a method to customize an eyewear design that is optimized tofit across various expressions:

In this embodiment, a) A computer system configured with an imagingdevice acquires image data and construct a model of the user's face at aneutral expression (using any methods previously described), b) Acomputer system acquires additional image data or the user with at leastone additional expression and constructs at least one additional facemodel (or obtain parameters necessary to adjust a single model tovarious expressions), and c) The computer system performs placement,design optimization, user adjustment, and preview with one additionalconstraint from previously described methods: The eyewear design,placement, and preview is performed across a plurality of face modelsrepresenting the user at multiple expressions. The optimal design thatsatisfied the constraints of all the face models or all the expressionsis produced, resulting in custom eyewear that is best fit to the useracross their range of facial expressions and movements.

Customization and Optics

As previously described in FIG. 1B, step 104, the computer systemprompts the user to enter his optical lens prescription information,which would be required to order prescription eyewear. The prescriptioninfo is also used to render lenses at the size, shape, and thicknessthat the user would receive in order to provide a more complete andrealistic preview of the entire eyewear. Since different prescriptionsrequire different optics (thinner or thicker lenses, more or lesscurvature), the user's specific prescription influences the visualappearance of their final product. If the user enters no data, anestimate of an average prescription lens or a plano lens (no opticalcorrection) is used for rendering, which will at least provide a view ofa lens in the eyewear frame. Alternatively, the user is asked generalquestions about their vision, such as near or far sighted, astigmatism,vision rating, type of lenses they prefer, etc. These generic questionscould be used by the computer system to associate with the most likelylens size and thickness for the user. The custom lens rendering isviewed by the user to judge whether a certain frame style and size isacceptable given the strength of his prescription, and/or whether thelens index he has selected is appropriate. For example, after seeing aprescription of −9.0 rendered with standard lenses that have a standardindex of 1.49 (resulting lenses would be thick), the user may prefer adifferent custom eyewear design that hides the thick lens edges or ahigher index of 1.67 or 1.71 to reduce the lens thickness. The computersystem also automatically suggests a lens index based on the framedesign and prescription to provide the best visual and aestheticappearance. For example, a user with a very strong prescription mayprefer a plastic frame due to the capability of its thicker rim tobetter aesthetically mask a thick lens edge, and the computer systemcould make that suggestion.

In an exemplary embodiment, the user may select lens styles, includingbut not limited to lens tint (clear, various shades of sunglasses,photochromatic lenses with estimates of tint indoors and outdoors,polarized lenses, etc), prescription style (piano, single-vision,digitally compensated, bifocal, progressive, etc), lens material index(1.5, 1.67, etc), lens coating(s), lens edge lenticularization (thinningof the lens edges), or brand. Any changes that are visible arerealistically rendered on the 3D eyewear model, including any distortionor optical effects that result from a particular lens type andprescription that an observer may see when viewing the user wearing theeyewear.

In an exemplary embodiment, more advanced measurements are derived fromthe quantitative anatomic model and eyewear model to enabledigitally-compensated (i.e. freeform), progressive, or other advancedoptical lens designs. In order to manufacture a digitally-compensatedand/or progressive lens, a variety of measurements are ideally required,including but not limited to pupillary distance, vertex distance,pantoscopic tilt, frame wrap, and lens height relative to pupils.Traditionally, eye care professionals (opticians, optometrist, etc) takethese measurements in-person utilizing specialized equipment or cameras.These measurements, even when done professionally are often difficult toestimate, such as measuring the distance from the surface of the eye tothe back to the lens. Measurements using anatomic and eyewear models ona computer system are much easier and more precise since there are notphysical barriers or limitations to taking a measurement. The user mayhave great advantage by obtaining the measurements automatically as theyselect their eyewear on the computer system, which eliminates the costand time resulting from a visit to an eye care professional.

In another embodiment, the product model is configured to optimize theoptical parameters used to make lenses. In addition to using the detailsand dimensions from the anatomic model to inform the lens design, theeyewear frame can be optimized to enhance the optical design. Forexample, standard vertex distance (distance from eye to inner surface oflens) is around 12-14 mm. With normal glasses, this can vary greatly,but configurable frames could be adjusted to achieve the optimalmeasurement. Other parameters include but are not limited to: framewrap, eye placement relative to the center of the lens, pantoscopictilt, etc. In this embodiment, a) A computer system obtains a scaledface model (using any previously described method) that has key facialfeatures identified, including but not limited to points, lines, andsurfaces of the eyes, nose, ears, brow, etc., b) The computer systemobtains a configurable 3D product model that has key featuresidentified, including but not limited to points, lines, and surfaces ofthe temples, nose pads, lenses, bridge, etc, c) The computer systemanalyzes dimensions of interest, including but not limited to vertexdistance, pantoscopic tilt, Pd, and frame wrap, d) The computer systemoptimizes the product model parameters, which changes the shape of theeyewear and how it rests on the user's face, until the dimensions arewithin their desired range (e.g. vertex distance 12-14 mm), e) Thecomputer system updates the configurable product model with newparameters, f) The computer system performs an optimization to obtain arigid transformation to align the product model to the face. The errorbetween key features of the product and face is minimized, and somefeatures is weighted more than others, and g) The computer systemtransforms the coordinates of the product model to align it with theanatomic model.

As described above, FIG. 7 illustrates some of the various measurementsneeded. Pupillary distance (Pd) is measured as binocular 703 a ormonocular 703 b measurements. Monocular measurements are often preferredto enable the best implementation of a user's prescription eyewear, butthey are harder to measure accurately and generally require physicalin-person measurement using specific equipment. Most Pd measurementsperformed on a single 2D frontal image of a user rely on a binocularmeasurement because it is easy for a system to detect the location ofthe eyes, while it is more difficult to accurately detect the exactcenter of the nose due to lighting constraints, the possibility the useris not facing the camera precisely, etc. However, the monocular Pd isbetter obtained by using the eye and nose features of the user'squantitative anatomic model. In this case, the additional informationprovided by the quantitative anatomic model allows the automaticdetermination of the center of the nose even if the individual 2D imagesused to construct the quantitative anatomic model were alone notsufficient to perform said measurement (for example, in no 2D image wasthe user perfectly facing the camera). If a straight-line distance weremeasured between the centers of the eyes, then the monocular Pd for eacheye would be defined as the distance from the eye center to the centerof the bridge of the nose. The vertex distance 709 is often hard tomeasure accurately in-person by a trained eyecare professional, but aquantitative anatomic model again provides an advantage. The vertexdistance is the distance from the center of the eye to the inner surfaceof the lens. An eye care professional has difficulty measuring thisparameter, on account of the difficulty of getting in-between the user'sframe worn on his face and his eye. In-person, the measurement needs tobe repeated for every eyewear design the user tries, which is quiteinconvenient and time consuming. Thus the measurement is oftenestimated. However, this challenging dimension is calculated with greatprecision by a variety of methods applied to the quantitative anatomicmodel of a user wearing the eyewear, such as tracing a ray from thecenter of the eye's surface on the to the inner surface of the lens onthe eyewear model. The perpendicularity of the ray relative to the planeof the face is ensured by using a variety of features in the model toconstruct a plane on the front of the face or by using the plane oflens. The pantoscopic tilt 710 is the vertical angle of the lens fromperfectly vertical. Again, this dimension is measured using thequantitative anatomic model coupled with the eyewear model. A plane isdefined through the lens and for the vertical position of the user'sface. The angle between the planes about a horizontal axis is used tocalculate the pantoscopic tilt. Frame wrap 704 is the horizontal angleof the lens positioned in the frame with respect to the user's face, andit is calculated with a similar method to the pantoscopic tilt by usingthe angle about a vertical axis. The fitting height 713 is calculated ina similar manner to the vertex distance. Once the computer systemcalculates lens position directly centered over the pupil (lens' opticalcenter), a dimension which is needed to calculate vertex distance, thevertical distance to the bottom of the inside surface of the lens holein the frame is calculated to determine the fitting height. All of thesemeasurements have the advantage of being performed with a 3D lenspositioned and previewed by the user relative to a quantitative anatomicmodel of the user.

In an exemplary embodiment, once the computer system has all theinformation necessary to manufacture a user's lens (all framedimensions, pupillary distance, additional face measurements if the lensis digitally-compensated, prescription information, lens material index,and choice of lens lenticularization), the system can also realisticallyrender the user's lens in the selected eyewear and positioned on theuser's image data. Algorithms for reconstructing a 3D version of a lensgiven the above information are well established and are necessary inorder to digitally surface and edge modern lenses. In an embodiment, thecomputer system uses sophisticated rendering techniques, such asrastering or ray tracing, to not only display the lens as a 3D object,but also render how light would be bent as it passes through the lens.Using said rendering techniques, the system can render a lens in a framepositioned on the user's face in order to allow the user to see exactlyhow they would appear as viewed by a third party. When the eyewear withlenses is placed on the user's face, an accurate distorted view of theface viewed through the lens can be shown. Moreover, the actualperformance of an anti-reflective coating is represented to the user, aswell as the appearance of lens distortion due to the inclusion of lensfeatures such as no-line progressive, bi-focal (dedicated magnificationzones), etc. With an accurate rendering in hand, a user is better ableto make an informed decision as to the type of frame and lens selected,with the tradeoffs of various choices made clearer. When a user shopsfor a lens in a retail environment, he is pressured to increase the lensindex material with the promise of a 20% reduction in lens thickness.But he has imperfect information; he is often not told how thick hislens will actually be in the frame he has selected, he often cannotoften visualize what a 20% reduction means in actual reduction of mm,and he often cannot make such a comparison in the abstract withoutseeing the aesthetics of the lens in person. This imperfect informationoften results in the user paying for an upgrade that he would not havedone had he been better informed: a 20% reduction may seem like a lotbut in actuality may only be a reduction of 0.7 mm and may not provideenough utility given the price. In this embodiment, not only can theuser be presented with a photorealistic rendering of the lens selected,but also all manner of lens configurations inside various frameconfigurations is presented and the user can make a more informeddecision. Moreover, the lens ultimately manufactured will look exactlylike the rendering, so there are no surprises.

In another embodiment, any lens configuration is displayed in across-section view such that the thickness of the lens in any locationcan be visualized and compared against other lens configurations(widths, index material, digitally compensated, etc)

Customization to Pre-Existing Eyewear

In another embodiment, the user captures image data of himself alreadywearing physical eyewear in his possession. The image data is capturedby the computer system or the user provides the image data to thecomputer system. The computer system analyzes the image data usingmethods similar to those previously described, but with additional imageprocessing to detect and determine the shape, color, and position of theeyewear. The computer system then adjusts configurable eyewear models tomatch the eyewear the user is wearing, similar to how the quantitativeanatomic model is adapted to the user's face. Shape models or otheralgorithms may be used to adapt and fit eyewear model to the image dataor features detected in the image data. This enables the user toduplicate, or duplicate and modify, eyewear they already possess. Forexample, a user may own a pair of eyewear they like with the exceptionof the frame color and width of the nose pads. The user may use thesystem to create a model of their eyewear, and adjust the frame colorand nose pad width using the methods and systems previously described.The user may also use this system to indicate where on his nose heprefers to wear existing eyewear (for aesthetic, utilitarian, or comfortreasons). The system will then place all new eyewear designs in thislocation on the user's nose. In another embodiment, the user uploads anyphoto of any person wearing eyewear, and the computer system may detectand analyze the shape and color of the eyewear, then update a new 3Deyewear model for the user that best matches the eyewear photo. Forexample, the user may have seen a photo of a friend or celebrity wearinga certain style of eyewear, and they may upload a photo to obtain asimilar design, which may then be further customized to their taste andanatomy.

In another embodiment, the eyewear designer or manufacturer provides asample eyewear frame that the user may wear during part of the imagedata acquisition process. Similar to the method described previously,the computer system detects and analyzes the eyewear. In thisembodiment, the advantage is that the eyewear model is a known size andshape to the designer. It's presence on the user's face in the imagedata provides both a reference scale for the data, since the size of thedetected eyewear is known, and it provides a very strong detectionfeature to enable more robust anatomic model reconstruction. By trackingthe known object in every frame and knowing it has consistentrelationships to other features of the user's face, the computer systemwill have more robust detection of the user's features. Additionally,the user would be able to physically touch and observe the quality andcraftsmanship of a sample eyewear frame.

Alignment

Referring to FIG. 1B at 108, the eyewear model is aligned with theanatomic model. In an exemplary embodiment, the configurable eyewearmodel and quantitative anatomic model are aligned based on anoptimization of reference geometry. The alignment may occur prior tocustomization to inform the customization process with information aboutthe geometric interaction between the user's anatomy and eyewear modelor after customization and prior to rendering to ensure the eyewearmodel is appropriately placed on the user's face. Ideally, eyewearshould rest with the nose pads tangent with and on the surface of nose,and temples on top of the ears and against the side of the head. The topof the eyewear should be a certain distance to the user's brow for agiven design. The eyes should be as centered as possible in reference tothe ideal eye location for a given design. Since there is no defaultplacement and each person's face is different, an approach for customeyewear must take into account the variable anatomy of each individualuser.

FIG. 10 shows two example eyewear designs: small round frames 1001 andlarge aviator frames 1002. The optimal eye locations for design 1001 areshown as 1003, well centered within the eyewear's lens opening; theoptimal locations for design 1002 are shown as 1004, off-center towardthe top of the lens opening. The ideal initial placement of the eyewearwould position the user's eyes as close as possible to (e.g. directlybehind) these locations.

An optimization is obtained by minimizing the distance between the:center of the eyewear and centerline of the nose; the top of eachmodeled ear at the location of the intersection of the head and thebottoms of the temples (which sit on the top of the ears); nose pads onthe eyewear and surface of the nose; center point of the eyes and thedesign's optimal eye location; pre-determined offset distance betweenthe brow and/or check bones and the specific eyewear front-frame.Alternative combinations of locations and measurements could also beused to optimize placement.

In an exemplary embodiment, the temples of the eyewear flex at thehinges to ensure a fit with the user's face by remaining in contact withthe sides of their face at the location above the ear where they makecontact. For example, if the width of the user's head at the ears werenarrower than the width of the eyewear, then the temples would bendinward to remain in contact with the side of this face so that the fitlooks realistic and the user can visualize if the eyewear is acceptableto him. The computer system represents the eyewear as a multi-partdynamic assembly or a flexible assembly that can allow for angularrotation of the temples defined by the hinges. In another embodiment,the temples themselves allow for elastic deformation, bending inward oroutward to ensure the temples are flush against the side of the head atthe tops of the ears. In this embodiment, the computer system mayrepresent the eyewear temples as a deformable unit that can safelyelastically flex a pre-determined amount.

In another embodiment, the relationship between quantitative anatomicmodel features and the eyewear model is set by machine learningtechniques and/or algorithms established from a database of trainingmodels, where the positions between the anatomic model and eyewearmodels have been set to optimal conditions. Based on new anatomicparameters and eyewear geometry, the system could assign an orientationand registration between the quantitative anatomic model and eyewearmodel using the classifier trained on the training data. This methodcould enable refinement of subtle user preferences for the placement ofeyewear.

Custom Product Previews

Once a quantitative anatomic model is established, scaled, andregistered to the image data and/or anatomic model, a representation ofeyewear is fit to the user's face. Referring back to FIG. 1A, 15 andFIG. 1B, 109, describe rendering the eyewear model on the user's imagedata to create a custom preview. In an exemplary embodiment, the user ispresented with his image data, with custom eyewear positioned correctlyand superimposed on his face. In an exemplary embodiment, thequantitative anatomic model is not displayed to the user, but is usedfor alignment and measurement data. The data is displayed as interactiveimages that the user can adjust, rotate, and zoom by interacting withthe computer system, including systems such as touchscreens, computerperipherals like mice, gesture interactions, or any other human computerinterface technology. This would enable the user to see custom eyewearon their face at various orientations.

In another embodiment, at least one still image is shown, such as afront and side view, or multiple views at set degrees around a verticalaxis centered on the users face. In yet another embodiment, an augmentedreality approach is used. A live video feed of the user's face is shownusing a computer system configured with a video camera. The quantitativeanatomic model tracks with the user's face in real time, allowing the 3Deyewear model to be displayed and superimposed on the user's face inreal time as the user moves his face in front of the computer system.This would create the illusion of looking in a mirror while trying onthe glasses, as one would in a retail store. In yet another embodiment,the user's image data may not be shown, and instead they are presentedwith a model of their face and head along with the 3D eyewear modelsuperimposed and correctly positioned on their face. Alternatively, theeyewear is represented as a series of pre-rendered images from variousangles rather than an actual 3D model. This method could enable easyimplementation of the display of high-quality pre-rendered images overnetwork systems.

In another embodiment, the analysis of image data is performed remotelyon another computer system such as a server(s) or cloud-computer(s) totake advantage of faster or more specialized or sophisticated computingcapabilities than the user's computer system may possess. Remote serverspossess thousands of networked CPU and GPU cores, with larger and fasterdata storage devices, yielding a system that is far more computationallypowerful and/or efficient than the local computer system in possessionof the user. The user's computer system transfer image data to theremote computer system, and after the image data is analyzed, thesolution or additional data such as rendered images is transmitted backto the user's computer system through a network or other datatransmission method. In another embodiment, the user's computer systemperform initial computations prior to sending data to a remote system orfinal calculations after receiving data back from the remote system,with the advantage that said initial or final computations reduce thequantity of data to transmit to or from the remote system, or reduce thecomputational burden on the remote system.

The computer system analyzes the user's image data for lightingintensity, quality, source, and temperature. Once a quantitativeanatomic model is constructed and registered to the image data, thecomputer system analyzes each individual image for at least one of thefollowing:

-   -   Color temperature within the bounds of the anatomic model with        reference to normal white balance.    -   Location of light and dark areas that correspond to highlights        and shadows, which can inform an analysis of lighting source. By        iteratively adjusting or directly computing a light source on        the anatomic model and minimizing the error between computed and        measured highlights and shadows, a lighting source or multiple        lighting sources is detected.    -   The overall brightness and contrast within the bounds of the        anatomic model informs the intensity and quality of the light        source.

The information on lighting is used to apply light sources to therendering of the 3D eyewear models to best match the image data,providing a near seamless integration of the eyewear model and user'simage data.

To achieve a realistic and flattering preview for users, there is anadvantage to setting a good white balance to the user image data suchthat the user appears to be in natural lighting with natural skin tones.Automatic white balance, as implemented in many image devices or imagepost-processing software, is used. Additionally, the detected face areais used to localize white balance information. There is furtheradvantage to having specific objects in the image to use for accuratewhite balance. Color tints of yellow, green, or blue are common fromdifferent lighting sources, and the adjustment should remove them. Inthis embodiment a) A computer system configured with a camera or imagingdevice is used to acquire image data of a user, b) A white balancetarget of known dimensions is positioned such that it is visible in atleast some images of the user, c) The computer system instruct the userto use a white or grey white balance target, such as paper, newspaper, aphone, phone case, electronic device. Or the white balance target is anobject with a known color, such as paper money, an electronic device, ora logo, d) The computer system detects the white balance target in atleast one image of the user, e) The computer system adjusts the whitebalance of the image data (e.g. rgb or color temperature and tint) untilthe target is neutral white or gray. And f) The computer system appliesthe white balance settings to all image data of the user.

The following embodiments describe systems and methods for creatingpreviews of custom eyewear on the user's image or anatomic data. Thequantitative anatomic model of the user's face is established, scaled,and registered to the image data such that the model coordinates andcamera position align the face model with the pose, position, and zoomlevel of the images of the user's face. The configurable 3D eyewearmodel is aligned with the quantitative anatomic model. Images arerendered of the configurable eyewear on the image data or models of theusers. The eyewear is rendered with a variety of techniques familiar tothose skilled in the art, including but not limited to raster, scanline, and ray trace rendering.

Embodiment to Render Image of Eyewear on User Image Data

In this embodiment, a) A computer system sets a camera position suchthat the anatomic and configurable 3D eyewear models are aligned withthe pose and position of user's image data, b) The computer system shows(or maintains) all surfaces of the configurable 3D eyewear model thatare positioned between the camera and the anatomic model, c) Thecomputer system hides (or deletes) all surfaces of the configurable 3Deyewear model that are positioned behind the anatomic model (eg, theanatomic model is between the camera and configurable 3D eyewear model),d) The computer system renders only the shown (or maintained) surfacesof the configurable 3D eyewear model, not rendering the hidden (orremoved) eyewear surfaces or the anatomic model, and e) The computersystem merges the rendered eyewear image onto the image of the user.

Embodiment to Render Image of Eyewear on User Image Data Using a DepthCalculation

In this embodiment a) A computer system sets a camera position such thatthe anatomic and configurable 3D eyewear model are aligned with the poseand position of user's image data, b) The computer system calculates thedepth (or distance) from the camera to all surfaces or vertices of theeyewear model and anatomic model at any given point in the image. Thecomputer system may store the depth values, c) The computer systemrenders only the closest surfaces at any given point or pixel on theimage, d) The computer system applies transparency to the anatomicmodel, such that it is not visibly rendered but is used in depthcalculations, and e) The computer system renders the eyewear on abackground consisting of the original image of the user.

Embodiment to Render Image of Eyewear on User Image Data with RayTracing

In this embodiment a) A computer system sets a camera position such thatthe anatomic and configurable 3D eyewear model are aligned with the poseand position of user's image data, b) The computer system sets thesurface of the anatomic model as invisible in the final rendering, butopaque and non-reflective to rays, c) The computer system traces raysbetween the camera and the scene, d) The computer system renders onlythe configurable 3D eyewear model, since the anatomic model isinvisible, e) The configurable 3D eyewear model is displayed with someparts hidden behind the opaque, but invisible, anatomic model, and f)The computer system merges the rendered image on the image of the user.The anatomic model may also be used as a surface that rays may castshadows onto.

Embodiment to Render Image of Eyewear on User Image Data with a Mask

In this embodiment a) A computer system sets a camera position such thatthe anatomic and configurable 3D eyewear model are aligned with the poseand position of user's image data, b) The computer system renders theconfigurable 3D eyewear model and anatomic model as a binary mask image(eg 1 for pixels where the configurable 3D eyewear model is positionedin front of the anatomic model and 0 for pixels where the anatomic modelis positioned in front of the configurable 3D eyewear model), c) Thecomputer system renders the configurable 3D eyewear model, d) The binarymask is applied to the rendered image, hiding the anatomic model and anyportion of the configurable 3D eyewear model that is behind the anatomicmodel, and e) The computer system merges the rendered eyewear image withmask applied onto the image of the user.

Embodiment to Render Image of Eyewear on User Image Data with a MaskDuring Render

In this embodiment a) A computer system sets a camera position such thatthe anatomic and configurable 3D eyewear model are aligned with the poseand position of user's image data, b) The computer system renders theconfigurable 3D eyewear model and anatomic model as a binary mask image(eg 1 for pixels where the configurable 3D eyewear model is positionedin front of the anatomic model and 0 for pixels where the anatomic modelis positioned in front of the configurable 3D eyewear model), c) Thecomputer system renders the configurable 3D eyewear model with the maskpreventing rendering in the black regions (the anatomic model andanything it is positioned in front of will not be visible or generatedduring rendering), and d) The computer system merges the renderedeyewear image with mask applied onto the image of the user

Embodiment to Render Eyewear with a Texture-Mapped Face Model

In this embodiment a) A computer system obtains a scaled face model of auser from image data (using any method previously described), b) Thecomputer system uses the images acquired to construct the face model tocreate a texture-mapped image of the user and apply the texture-mappedimage to the face model, c) The computer system positions a configurable3D eyewear model to be aligned with the face model of the user (usingany method previously described), d) The computer system renders thetexture-mapped face model and configurable eyewear model together tocreate preview image data for the user, e) Optionally, thetexture-mapped face model and eyewear model rendering is superimposed onthe original images of the user or f) Optionally, the computer systemallows the user to provide input to control or adjust the pose andposition of the face and eyewear model, rendering the image data aftereach adjustment by the user.

Previews Using User Photos

It is desirable for a user to see previews of custom eyewear on anyphoto they choose. The image could be a favorite photo, professionalphoto, or other image that is different from the images used to buildthe anatomic model. This embodiment describes a method to align theanatomic model with a new image and then render the eyewear on the newimage. In this embodiment a) A computer system obtains a new image of auser (not necessarily used to obtain anatomic data). The image isuploaded, linked to the computer over a network connection, sent viaemail, sins, or other communication systems, etc, b) A computer systemobtains a scaled face model of a user from image data (using any methodpreviously described), c) The computer system detects the face, estimatepose, and detect facial features in the new image, d) The computersystem performs a rigid transformation of the face model and camera toalign the face model features with the new image detected facialfeatures, e) The computer system positions a configurable 3D eyewearmodel to be aligned with the face model of the user (using any methodpreviously described), and 0 The computer system renders the eyewear onthe new image of the user (using any method previously described)

Simulated Camera Perspective

It is also desirable to simulate camera or vision properties (focallength, distortion, field of view, distance from subject) that aredifferent than the camera used to acquire the image data. The user maywant to simulate the perspective of human eyes or of a more flatteringcamera lens. When compared to human eyes or cameras at furtherdistances, computer camera wide angle lenses that take photos at shortdistances often accentuate and enlarge objects closer to the lens (noseor glasses) and reduce the appearance of objects further from the lens(ears and side of head).

Referring to FIG. 25: a) A computer system obtains a scaled face modelof a user 2501 from image data (using any method previously described),b) A computer system positions a configurable 3D eyewear model 2502 tobe aligned with the face model of the user (using any method previouslydescribed) c) A computer system sets a camera position 2503 such thatthe anatomic and configurable 3D eyewear models 2504 are aligned withthe pose and position of user's image data, d) A computer system altersthe intrinsic camera parameters and distance 2505 from the model tosimulate different perspectives and camera properties, while stillmaintaining the same placement of eyewear aligned with the user's imagedata 2506, e) The computer system renders the eyewear on the image ofthe user (using any method previously described), and f) Optionally, thecomputer system uses the anatomic information as seen from the originaland simulated camera properties and position to deform and distort theoriginal user images. The distortion could allow the underlying imagedata to better represent a different camera perspective.

Embodiments for Physical Previews

It is advantageous to have a physical preview of a custom productinstead of a digital preview. The following embodiments describe twomethods to provide a user with a physical preview of their eyewear:

In this embodiment a) A computer system obtains a scaled face model of auser from image data (using any method previously described), b) Acomputer system customizes a configurable 3D eyewear model to fit theface model of a user (using any method previously described), and c) Acomputer system converts the 3D eyewear model into a digital file forrapid manufacturing. Techniques include but are not limited to:

-   -   i. directly 3D printing the eyewear model with plastic, paper,        or metal. The model is converted to a hollow body to save cost        and weight.    -   ii. converting the 3D model into a flat pattern and cutting a        flat sheet (paper, cardboard, plastic, metal, etc) with a CNC        laser, waterjet, vinyl cutter, mill, etc. Optionally, folding or        bending the flat sheet.    -   iii. Converting the 3D model into multiple pieces, such as frame        front and temples, that are produced Using the methods        previously mentioned. Assembling the pieces using fasteners,        glue, or other methods.        d) A computer system receives an input from the User, including        but not limited to: name and address, optional payment        information, other contact information, shipping preferences,        and e) A computer system generates instructions to build,        package, and ship a rapid prototype of the custom eyewear model        to the user.

In this embodiment a) A computer system obtains a scaled face model of aUser from image data (using any method previously described), b) Acomputer system customizes a configurable 3D eyewear model to fit theface model of a user (using any method previously described), c) Acomputer system converts the 3D eyewear model into a digital file forrapid manufacturing. Techniques include but are not limited to:

-   -   i. directly 3D printing the eyewear model with plastic, paper,        or metal. The model is converted to a hollow body to save cost        and weight.    -   ii. converting the 3D model into a flat pattern and cutting a        flat sheet (paper, cardboard, plastic, metal, etc) with a CNC        laser, waterjet, vinyl cutter, mill, etc. Optionally, folding or        bending the flat sheet.    -   iii. Converting the 3D model into multiple pieces, such as frame        front and temples, that are produced Using the methods        previously mentioned. Assembling the pieces using fasteners,        glue, or other methods.

d) The computer system generates files for the user and provide a meansfor the user to obtain the digital files, including but not limited toan email, link to download from a network server, attachment to adigital message, etc., and e) The computer system generates instructionsfor the user to build the rapid prototype with the files, such asinstructions to use a printer or 3D printer, instructions for assembly,instructions for sending the file to a service to be printed or build,etc.

Embodiment to Render a Life Size 1:1 Image of the Eyewear

The user may want to understand the true size of their eyewear inaddition to a preview rendering of the eyewear on their images or model.For example, the user could compare the size to existing eyewear theyown.

In this embodiment a) A computer system obtains a scaled face model of aUser from image data (using any method previously described), b) Acomputer system customizes a configurable 3D eyewear model to fit theface model of a user (using any method previously described), c) Acomputer system obtains information about the display of the computersystem, such as resolution, pixel size, overall display dimensions. Thecomputer system obtains this information from itself, from a webbrowser, from the user providing information about the display orcomputer system model, d) A computer system calculates the pixel size ofthe display (for example, by dividing the length and width of the screenby the number of pixels), e) A computer system renders the eyewear modelin various orientations, such as front view, side view, top view with areal-life scale of 1:1 by using the pixel size and dimensions of theeyewear model, f) A computer system displays the 1:1 images to the user,and g) Optionally, the computer system renders a real-time interactivegraphic of the eyewear model that the User can control through an inputdevice to rotate and pan in real-life 1:1 size.

Physics Based Preview

A common problem with eyewear fit is the nose and temple sizes beingincorrect, resulting in eyewear that slips down the nose of the user. Aphysics-based preview method can simulate if eyewear will stay on thenose. The following is an embodiment for physics-based adjustment:

In this embodiment a) A computer system displays a preview of a customeyewear model on a user's image data and face model (using any methodpreviously described), b) A computer system accepts user input (touchscreen, slider bar, mouse control, gesture, etc) to move the front frameof an eyewear model vertically up or down with respect to the user'sface and/or move the front frame closer or further from the user's face,c) A computer system enforces constraints to ensure the eyewear does notinterfere with the model, such as the nose pads intersecting the surfaceof the face model or the temples intersecting the top of the ears of theface model, d) A computer system applies the following physicalproperties to the eyewear model and face model

-   i. Mass of eyewear model, which is estimated from its volume and    material properties-   ii. Coefficient of friction of eyewear material-   iii. Coefficient of friction of skin, which is estimated as a    general property for human skin    e) A computer system solves a system of mechanics equations    representing the balance of forces between gravity acting on the    mass of eyewear and the opposing frictional force of the eyewear    nose pads contacting the face model nose surface and the eyewear    temples contacting the face model ears, and f) The mechanics    equations are iteratively solved until a steady state is reached    where the eyewear is positioned with balanced forces supporting it.

Lens View Rendering

In another embodiment, the computer system simulates the vision of theuser when wearing progressive eyewear. The user is presented with a viewsuch that he can look through his configured lens and see the world ashe would see it through the lens. This technique is best applied to thecustom configuration of no-line digitally-compensated (freeform)progressive lenses. A photo can be displayed on the screen (pre-selectedor user-uploaded or a live image stream from the computer system imagingdevice) with the lens positioned in front of the image. Information issuper-imposed over the lens identifying the various corrected regions ofthe lens to the user (areas with distortion, corridor, areas of maximummagnification, transition areas, etc). The system can display how faraway it has virtually positioned the photo behind the lens, and usingray tracing rendering techniques known to those in the art, the photocan be distorted as the light passes from the photo through the lens andto the viewer. Changes to the lens design or shape/size of the eyewearcan update in real-time in this preview. A user would be able to betterunderstand the areas of the lens that would be distorted (peripheralareas in a progressive lens), and the amount of distortion given variousdigital lens designs. In another embodiment, the computer system usesits imaging sensor to provide a live preview of what it sees through thesystem display, and the computer system may distort this view inreal-time given the lens design selected. This live-previewaugmented-reality view would allow the User to experience life as seenthrough the lens they have customized given lens parameters and customframe parameters.

User Interaction and Control

Referring to FIG. 1A at 16 and FIG. 1B at 110, 113, 114, the computersystem provide a means for the user to interact with the computer systemfor shopping, selecting, editing, modifying, previewing, controlling thepreview, visualizing, purchasing, and performing other activitiesrelated to customizing a product.

FIG. 11 shows an example computer system interface 1101, which would beshown on the display of the computer system, with a preview of eyewear1106 on user 1102. The computer system interface contains controls 1103for ordering, viewing, configuring, sending previews, sharing, obtaininghelp, or other functions. The eyewear style or base design is selectablewith controls 1108 and colors/finishes with controls 1105. Instructionsare provided to the user through the display of 1104. It should berecognized by those skilled in the art that a variety of other designscould suite the same needs described for viewing, customizing, andordering eyewear. For example, multiple views of the eyewear may beused, such as 2, 4, or 9 windows displayed with different styles at thesame time or different view perspectives of the user. In one embodiment,the computer system displays multiple instances of the user, with eachinstance wearing a different configuration of custom eyewear. Eacheyewear shown may have one or a plurality of options changed. Forexample, the display shows nine instances of the user's face, with eachinstance showing the user wearing the same custom eyewear design buteach design is displayed with a different color, style, or lensmaterial. In another example, multiple instances of the ⁻user isdisplayed, with each wearing the same style and color of eyewear butautomatically sized to the face slightly differently, such as slightlylarger or smaller variations or altering the eyewear placement slightlyhigher or lower on the face (using the same sizing algorithm or aplurality of competing algorithms). In another example, the displayshows multiple instances of the user wearing the same or differentcustom eyewear as viewed from a different angle (front, isometric, side,top). As one instance of the user is manipulated, all instances updatesimultaneously. For example, as the user changes the view of oneinstance, the same change of view is applied to all instances.

In an exemplary embodiment, the computer system allows the user toadjust the position of the eyewear model on his face. The user selectsthe eyewear with their input device and adjust it at certain locationsby moving, dragging, or making other controlling actions with the inputdevice. For example, the user grabs the eyewear at the temples and slidethem up or down to fit better onto the ears, or he grabs it at the nosebridge to place or adjust how and where it sits on his nose.Additionally, the user is able to correct any errors in the automatedplacement of the eyewear.

In another embodiment, the eyewear model is adapted and configured inreal-time or near real-time as the user makes adjustments to theposition. For example, typically one would simply moving the eyewear toa new position for previewing, which may result in the eyewear no longerfitting while in that position because the nose may be too narrow ortemples too long or some part may not fit based on the new position.With configurable eyewear, the model is adapted as the user moves it,such that the eyewear changes shape to fit the user's face in the newposition. If the user pulled the eyewear away from their face, the nosepads would slightly lengthen and the temples would slightly lengthenamong other changes, as opposed to the nose pads being too short andtemples too short, and the glasses falling off the user's face withoutadjustment.

For example, in FIG. 11, the eyewear 1106 on user 1102 previewed withinterface 1101 is positioned at an incorrect angle. The user adjusts thepositioning by selecting eyewear 1106 with an input device and moving itin the direction 1107 shown. As shown in view 1109, the preview then isupdated to show eyewear 1110 properly positioned on the user's face perthe user's specification. Alternatively, the user is able to manuallyidentify specific points where the ears, eyes, nose, and other featuresare so the computer system can align eyewear more accurately. It iscommon that a person's left and right ear is at different heights,usually causing eyewear to sit crooked or angled. The ability adjust theangle and ensure that the custom eyewear design accounts for thediffering heights of the left and right ears provides a great advantageto the user obtaining and proper and comfortable fit. With aconfigurable eyewear model, the proper fit can not only be displayed forpreview, but actually configured and manufactured so the user gets aproduct that fits in real-life as well as it looks on preview, adistinct advantage over prior art.

Once an eyewear model is automatically placed on an anatomic model of auser, it is desirable to allow the user to adjust placement to theirpreference during preview. For example, the user may like to wear theirglasses higher or lower with reference to their eyes or nose or furtheror closer to their face. These adjustments can help to inform a customeyewear design that is fitted to position the eyewear to the user'spreference. One of the great advantages of fully custom eyewear is thatthe underlying design can be adapted to fit a user's placementpreference. Typically a user could preview or wear eyewear at differentpositions on their face (closer or further from the eyes or higher orlower on the nose), but if the eyewear is not the right size and shape,then it will be uncomfortable, not stay in position, or not be possibleto wear at the desired position. The following embodiments describesystems and methods to enable custom placement of custom eyewear:

Embodiment to Adjust the Vertical Position of an Eyewear Model on aUser's Face by Setting Vertical Position and Adapting Eyewear ModelPlacement

In this embodiment, a) A computer system displays a preview of a customeyewear model on a user's image data, b) A computer system accepts userinput (touch screen, slider bar, mouse control, gesture, etc) to movethe front frame of an eyewear model vertically up or down with respectto the user's face, c) The computer system solves a system ofconstraints to properly adjust the eyewear model on the user's face.

-   -   i. The front frame vertical height must be in the vertical        position specified by the user    -   ii. The temples of the eyewear must contact the top point where        each of the user's ear and head intersect of the face model. The        temples is adjusted to different heights depending on symmetry        or asymmetry of the user's face    -   iii. The nose pad regions of the eyewear must contact but not        intersect the user's nose of the face model    -   iv. Optionally, the system of constraints could be other points,        lines, surfaces, or features as previously described.

d) If the constraints can be satisfied by adjusting the eyewear positionto achieve the user-specified vertical position of the eyewear model,then the system will display an updated preview with the new eyewearmodel position, and e) Optionally, if the constraints cannot besatisfied, the system informs the user that the position is not possibleor that they eyewear may not fit properly (e.g. slip down nose).Alternatively, if the calculation is done in real-time, the user willonly be able to adjust the eyewear within a set range of verticaldistances.

Embodiment to Adjust the Position of an Eyewear Model on a User's Faceby Setting Position and Adapting Eyewear Model to Achieve the DesiredPosition

In this embodiment a) A computer system displays a preview of a customeyewear model on a user's image data, b) A computer system accepts userinput (touch screen, slider bar, mouse control, gesture, etc) to movethe front frame of an eyewear model vertically up or down with respectto the user's face and/or move the front frame closer or further fromthe user's face, c) The computer system solves a system of constraintsto properly adjust the eyewear model on the user's face,

-   -   i. The front frame vertical height and closeness to the face        must be in the position specified by the user    -   ii. The temples of the eyewear must contact the top point where        each of the user's ear and head intersect of the face model. The        temples is adjusted to different heights depending on symmetry        or asymmetry of the user's face    -   iii. The nose pad regions of the eyewear must contact but not        intersect the user's nose of the face model    -   iv. Optionally, the system of constraints could be other points,        lines, surfaces, or features as previously described.

d) If the adjustment creates a gap or interference between the eyewearmodel and user's nose in the face model, then the nosepiece of theeyewear model is adapted by the computer system (adjust thickness,position of pads, width, etc) to create a contact with the user's nose.e) If the adjustment creates a gap or interference between the templesand the user's ears of face, then the temples is adapted by the computersystem (adjust length, angle, etc), f) If the adjustment creates a gapor interference that is outside the solvable domain of the customeyewear model constraints or if large portions of the eyewear causeinterference (eg entire frame moves into the face), the computer systemdoes not allow adjustment to the unacceptable position, and g) Thesystem displays an updated preview with the new eyewear model position

Embodiment to Adjust the Position of an Eyewear Model on a User's Faceby Pre-Computing a Series of Options

In this embodiment a) A computer system calculates the optimal fit of aneyewear model on a user's image data, b) A computer system creates aplurality of adjustments to the vertical position of the eyewear, movingit up and down the nose or further/closer to the face in set incrementsfrom the optimal position (ie, +4 mm, +2 mm, −2 mm, −4 mm), c) Acomputer system pre-renders images of the user with the eyewear model inall the adjusted configurations, d) A computer system displays a previewof a custom eyewear model on a user's image data, e) A computer systemaccepts user input (touch screen, slider bar, mouse control, gesture,etc) to move the front frame of an eyewear model vertically up or downwith respect to the user's face in the increments that were used topre-compute the adjusted configurations, and f) The computer systemdisplays the adjusted configuration rendering that matches the usersselection

Embodiment to Adjust the Vertical Position of an Eyewear Model on aUser's Face with Surface Constraints

In this embodiment a) A computer system calculates the optimal fit of aneyewear model on a user's image data, b) A computer system setsconstraints that limit the potential movement between the eyewear andface models,

-   -   i. The eyewear model only moves in certain directions (e.g.        further/closer to the face or vertically up and down)    -   ii. The eyewear model only rotates along an axis formed by a        line through the contact point between each ears and temples    -   iii. The eyewear model must maintain contact between the temples        and the top point where each user's ear and head intersect on        the face model    -   iv. Both eyewear model nose pads must be in contact or within a        tolerance of the nose surface on the face model    -   v. Optionally, the system of constraints could be other points,        lines, surfaces, or features as previously described.

c) A computer system displays a preview of a custom eyewear model on auser's image data, d) A computer system accepts user input (touchscreen, slider bar, mouse control, gesture, etc) to move the eyewearmodel. The computer system calculates the system of constraints witheach user input, e) The eyewear model only moves within the predefinedconstraints, and f) The computer system displays the eyewear modelposition adjustment as it is moved by the user.

Embodiment to Adjust the Vertical Position of an Eyewear Model on aUser's Face with an Image of Their Current Eyewear

A user may already possess eyewear that sits on their face in a positionthey prefer. This embodiment describes how a new custom eyewear isdesigned such that the same positioning is obtained, even if the eyewearstyle, shape, and design are different:

In this embodiment, a) A computer system configured with an imagingdevice acquires image data and construct a model of the user's face(using any methods previously described), b) The user uses the computersystem to acquired image data of the user wearing a eyewear positionedto their preference, c) The computer system extracts anatomic locationsof where the eyewear contacts the user's face (eg where the nose padsrest relative to the user's nose) and/or reference positions of wherethe eyewear is located with respect to facial features (eg the top ofthe eyewear is positioned a certain distance above the eyes or thedistance down the length of the nose where the eyewear bridge ispositioned), d) The computer system uses the anatomic locations and/orreference positions to optimize the fit and design of new customeyewear, e) The computer system solves a system of constraints toproperly adjust the eyewear model on the user's face.

-   -   i. The front frame vertical height, angle, and closeness to the        face must be in the position closest to the extracted data    -   ii. The temples of the eyewear must contact the top point where        each of the User's ear and head intersect of the face model. The        temples is adjusted to different heights depending on symmetry        or asymmetry of the user's face    -   iii. The nose pad regions of the eyewear must contact but not        intersect the User's nose of the face model    -   iv. Optionally, the system of constraints could be other points,        lines, surfaces, or features as previously described.    -   f) A computer system displays a preview of a custom eyewear        model on a user's image data

User Interaction and Control of Configurable Model

A great advantage of a custom eyewear system is the ability for a userto directly modify and update the product to their preference. In anexemplary embodiment, the computer system provides the user with controlto edit or adjust the eyewear shape from the base design, which servesas a template for modification. The base design may have already beenautomatically customized for the user by the computer system or it maybe the original base design prior to any customization.

FIG. 12 shows an example computer interface 1201 for adjusting eyewear1203 previewed on user 1202. The base designs consist of a variety ofstyles or materials, including but not limited to fully-rimmed,semi-rimmed, rimless, plastic, or metal. The controls include but arenot limited to: control points on the eyewear that can be dragged oradjusted, sliders that are linked to certain features, directly drawingon the frame, and touch, gesture, mouse, or other interaction to stretchor push/pull features of the frame. In one embodiment, the controlsallow the user to change certain limited features, including but notlimited to the nose pad width, the temple length and height, and thewidth and height of the front of the eyewear. For example, if user 1202in FIG. 12 has a narrow face, he adjusts the eyewear 1203 to make theoverall size of the eyewear narrower. The user selects the eyewear 1203with the computer system input device, and moves the edge of the eyewearinward toward his face as indicated by the arrow in FIG. 12. Theresulting modified eyewear 1206 is shown in the updated preview 1205.The ability for the user to make such easy and custom adjustments toeyewear before purchasing represents a major change in the way theeyewear products are purchased from the current state of the art. Thefeedback may be nearly instantaneous, with the user seeing the renderedpreview updated on the computer system display.

In one embodiment, constraints are used to limit the customizationwithin bounds that are predefined with the configurable model. Theparametric design and constraints of the model may be used to limitfeature adjustment to preserve each eyewear's base design while makingthe process simple for the user to achieve custom fitting and sizing.While some use cases may have advantage of giving the user 100% controlover the design, there is a distinct advantage to limiting theadjustment so the user can easily obtain an aesthetically pleasing andmanufacturable product. For example, without any constraints, the usermay accidentally make a self-intersecting or highly assymettrical orjagged, unappealing design that would neither fit nor look good. Inaddition to built-in constraints, controls such as control point,arrows, etc) may be highlighted only on the areas that are adjustable,or they highlight as the user moves their input device over the areas,or there are instructions explaining what portion(s) of the eyewear theycan alter.

In another embodiment, the user has fewer limits in what he can adjustwhile still preserving the overall eyewear design. For example, thecomputer system enables the user to grab and adjust any part of theeyewear, giving controls to adjust length, height, width, and thicknessof any portion of the eyewear, as well as the curvature of variousmembers such as the rims and temples. FIG. 13 illustrates an examplebase eyewear design 1301. A user directing a computer system inputdevice selects a point on the eyewear at 1305 and move along the dottedline in the direction of the arrow 1306 to point 1307. The eyewear 1302would then be modified in the region 1308 that was edited. To retainsymmetry while simultaneously reducing the number of steps necessary tocustomize eyewear, a change on one side of the eyewear is equallyapplied to the other side of the eyewear, as shown in updated eyewear1303.

User Adjustments Without Direct Editing

In another embodiment, the computer system may ask the user questions tohelp guide him to or through adjustments. For example, the computersystem may ask, “Is the eyewear currently too wide or narrow on yourface?” or “Is the eyewear currently too thick or thin?” or “Do youprefer larger or smaller styles?” The user would be able to select anoption or answer the prompts through the interface and then subsequentlyobserve an adjustment to the eyewear in response. When coupled withmachine learning techniques described herein, this could represent apowerful means to provide a personalized and custom recommendation,while allowing slight adaptation based on live feedback from the user.

In another embodiment, the computer system alerts the user to certainkey areas to adjust, including but not limited to the nose pads andtemples. The nose and top of both ears are the three key contact pointsthat must fit well, and each ear may be at a different height. Thecomputer system may ask the user to inspect these particular areas andadjust as needed. For example, the user may adjust the length of thetemples until they fit well over the ears, or adjust the temple anglesindependently to correspond to his differing ear heights such that thefront frame of the eyewear sits ideally and aesthetically level on hisnose.

In another embodiment, the user may adjust, modify, reposition, orselect new eyewear designs in real-time on a preview of their imagedata. As previously described, a real-time preview is provided, and theuser is given control over modifying the eyewear design in real-time.

Improper Fit

Referring back to FIG. 1B, step 111, describes the computer systemdetecting when a potentially improper or uncomfortable fit exists or ifa design has been created that is not possible to order. Theseundesirable configurations may result from the user's interaction andcustomization of their model, and they may not be aware of how theirchanges affect the model. For example, if the temples are required toflex too far to accommodate the user's face, they are uncomfortable dueto the pressure applied to the sides of the user's head. The pressure onthe user's head is calculated based on the hinge design properties, thedegree of hinge and/or temple deformation, and the distance from thehinge to where the temples contact the user's head. In another example,the nose pads are too tight on the nose or too lose and the eyewear mayslip. There may be an absolute interference than can be detected by thecomputer system. An analysis of the anatomic model and configurableeyewear model can detect surfaces that interfere. The pressure on thenose pads is calculated based on the face and eyewear geometry and thematerial properties of the eyewear. A warning or automatic adjustment tothe design is provided if the pressure is determined to be too high.Additionally, the lenses may be positioned at a non-optimal angle suchthat the user would have a poor visual experience or sub-optimal visualacuity. The computer system analyzes the following criteria, amongothers, between the 3D eyewear model and the quantitative anatomic modelto ensure a proper fit on the user: Interference or gap between the nosepads and nose, Interference or gap between the top of the ears andtemples, Angle of temples (inward or outward) needed to fit to the ears,Angle of lenses, and Position of eyewear on nose and position of eyesrelative to lenses (e.g. are the eyes well centered within the lenses?)

The computer system couples the dimensional information with materialproperties, force and deformation calculations, and computationalsimulation of stress/strain. Specifications may exist for each metricanalyzed and if a criterion is not met, the user is alerted.Alternatively, the computer system automatically suggests an alternativeor set of alternatives.

Custom Finishes

In an exemplary embodiment, the computer system provides the user withcontrols to change the color, finish, texture, or material of theeyewear. The user's control of these options may occur without automatedrecommendations from the computer system or they user may be givencontrol after the computer system makes an initial custom design. Thecomputer system displays a plurality of colors that is previewed on orapplied to the eyewear. The user selects different colors for variousportions of the eyewear. The color selection may be limited to a set ofcolors/finishes established by the manufacturer or there is a pluralityof hundreds, thousands, or more colors/finishes. The user also selectsoptions for material finish to preview. Examples of finishes that isselected and rendered include polished, brushed, satin, clear coat,gloss, matte, embossed, hammered, grained, etc. User changes and editingof the eyewear may happen in an editing interface with updates appliedto the preview view, or said changes and edits are applied and previewedin real-time.

In another embodiment, the user take a photo of an object such asclothing, nail polish, pictures, etc. The user provides the photo as adigital image or uses the computer system to take the photo. The userselects a point or region of the photo for the computer system to matchthe color or pattern. The photo is analyzed by the computer system and acustom color or pattern is specified from that image. The computersystem may require a calibration standard to be employed to obtain highaccuracy in color matching and reproduction. The calibration standard isa printed card with a variety of calibrated colors and shades on it thatthe user must include in the image. The manufacturer may supply thiscard to the user, or the user prints it. The computer display may alsobe presented next to the object with a color that is desired. Thedisplay may have a color calibration pattern displayed on it, whichcould be captured along with the object in a mirror or using a secondimage-capture device. Alternatively, the user is prompted to include aknown object in the photo. The known object would be an item that wascalibrated and stored in the computer system. Examples may includeubiquitous logos that are known to be professionally-printed with ahigh-degree of color accuracy and consistency, such as a logo on a foodbox or magazine, soda cans, currency, or credit cards. Alternatively,the computer system may have a database of known colors from othermanufactures, such as makeup, paint samples, automobiles, or fabrics—auser is able to select the color of her favorite shirt, car, or nailpolish color from said database and the manufacturer would then have thecolor information necessary to accurately reproduce and match theintended color.

In another embodiment, the eyewear is customized with a pattern, image,or text from the user. The pattern, image, or text will herein bereferred to as pattern. The pattern is printed, engraved, etched,painted, or otherwise applied to any surface of the eyewear. The patternis generated from a library of available options on the computer system,provided by the user from her image similar to the previous descriptionof custom colors, or entered by the user. For example, the user may wantto print his name inside the temples. Or he may desire to etch a designof lines on the side of the temples or print a textured pattern ofleaves on the eyewear. The pattern is rendered and previewed to the useron the 3D eyewear model, and subsequently accurately reproduced on themanufactured eyewear.

In another embodiment, the eyewear is customized with accessories,including but not limited to logos, charms, jewels, etc. For example, abase design may have an option to place an accessory on each temple nearthe hinge. There is a default accessory, and the user may elect tochange, reposition, or remove it. The user may select from a pluralityof options including a variety of shapes, colors, materials, etc. Theaccessories are rendered by the computer system to display on the 3Deyewear model for the user to preview.

Preference Records

In an exemplary embodiment, once the user has selected eyewear andadjusted its size, color, and other features, these preferences arerecorded and stored to a non-transitory computer readable media. Theuser's models, image data, and other information are also stored by thecomputer system. When the user selects alternate eyewear designs, suchas a different material or different style, the eyewear is adjusted totheir preferences based on their past interactions and preferences,therefore making the experience of browsing through eyewear morecustomized while also reducing repetitive tasks. For example, one thedesired fit preferences are established, any design or style can beupdated to fit the user according to their preference. If they likeeyewear that is slightly smaller than the width of their face and theylike to wear it further from their eyes, then all the styles could beadjusted to that preference. In another embodiment, the preferences fora specific user are refined as he uses the computer system. Aspreviously described in the method to build the training database ofpreferences, the computer system records and track a user's preferencesas he shops and previews eyewear. This information is used to refine hispreferences and add to the information he entered or was previouslyanalyzed from his supplied image data. The user's stored preferences mayalso be used to build a larger database for future prediction andcustomization of new users, as mentioned previously.

As the user and/or computer system adjusts the eyewear, the magnitudeand direction, when relevant, of the change is recorded by the computersystem. The configurable eyewear model is updated by adjusting theappropriate model parameter by an amount to match the change requestedby the user. Any constraints programmed into the model are checked andif a limit is exceeded, then the computer system provides a warning tothe user. Alternatively, the change is applied up to the limit and anyexcess change beyond the limit is ignored or disallowed (with or withoutwarning the user of the limit exceeded). For example, if the userchanges the width of the eyewear from 140 mm to 190 mm, but the maximumdesign width is limited to 170 mm, then the eyewear would adjust only tothe maximum 170 mm, and the user is notified of reaching this limit. Anupdated model is rendered and displayed by the computer system aspreviously described such that the user can preview the new 3D eyewearmodel on his image data. In another embodiment, the changed area of theeyewear is highlighted or identified to the user for a period of time oruntil he accept the change. The user is provided with a provision toundo (or redo) any changes he requested.

Efficiency of Configuration

As users or the computer system request changes to the configurablemodel to fit different users, it may desirable to have a plurality ofcustom designs that are preconfigured for efficiency. For example,hundreds, thousands, or millions of configurations of a design could bepre-configured and stored on a computer system or network-accessiblecomputer system. If these pre-staged configurations span the mostcommonly accessed design configurations, then they can be quicklyaccessed and displayed to the user. Alternatively, a shape matchingalgorithm, look-up table, or other techniques are used to find the modelthat is closest to the user's preferences. Subsequent minor adjustmentsare then made from the pre-staged configuration to fine tune theconfigurable model to the exact user preferences.

Preparation for Manufacturing

As illustrated in FIGS. 1A at 17 and 1B at 115 and 116, the computersystem stores data to represent the user's preferences and designs, andsubsequently calculates a price and shipping estimate. After a userdetermines the final custom eyewear he wants to order, the computersystem may generate a final representation that is morephoto-realistically rendered and of higher quality and resolution if theoriginal preview images were made to a lower quality for efficiency. Thecomputer system provides to the user a price, expected shipping date,and other information prior to the completion of the order for hiscustom eyewear. The representation may consist of the various parametersand settings selected by the user or a final 3D model of the eyewear.The computer system transfers the eyewear representation andpreferences, dimensions, configuration data and other information via anetwork connection or other means of information transfer to anothercomputer system accessible by the manufacturer. In addition to theeyewear representation, the computer system may also receive the user'spersonal information, payment details, shipping address, image data, andany other information needed to complete the order.

In order to provide an estimated shipped date and price, the computersystem actively tracks a number of parameters, including but not limitedto: an inventory of all raw materials needed, current productioncapacity, work in progress, future schedules, orders scheduled, and leadtimes on materials or production capacity, etc. The computer systemperforms scheduling and shipping estimates to provide the user with anexpected delivery date or provides the manufacturer with actions neededto achieve a guaranteed delivery date for the user.

Manufacturing Custom Products

FIG. 1B at 114 illustrates the user's decision to purchase eyewear. FIG.1A at 18 and FIG. 1B at 116 and 117 describe analyzing and preparinginformation and files for eyewear and lens manufacturing. The finaleyewear representation, preferences, dimensions, configuration data,once in the manufacturer's computer system, are analyzed to create botha manufacturing work order and set of manufacturing CAD, CAM, CNC, orother manufacturing and modeling files automatically. A serializedidentifier linked to the user's order is created to track the eyewear asit moves through the production process. The computer system associatesthe serial number with raw materials, specifications, or qualitychecklists. The computer system also prepares manufacturing filesdepending on the method of manufacture needed for the particular eyewearmodel, including but not limited to: model files for rapid prototypingor additive manufacturing methods; model files converted into tool-pathCNC code for machining (e.g. g-code), routing, milling, or othersubtractive manufacturing methods; model files converted into flatpatterns for photo-etching; model files converted into flat patternswith tool-path or robotic control code for laser-cutting;laser-marking/etching, water jet cutting, stamping (and stamp toolproduction), punching (and punch tool production), or other 2-D cuttingmethods; model files converted into rapid prototyping or additivemanufacturing methods of an inverse geometry to create a mold forinjection molding, casting, or other tool production, and model filesconverted into robotic control instructions for part handling,polishing, assembly, drilling, cutting, etc.

The computer system also prepares manufacturing files depending onprescription information, lens material, and user information convertedinto lens surfacing, lens laser-marking, and lens edge machininginstructions for lens manufacturing; Parameters entered by the user forupdating existing manufacturing files for any of the above-mentionedmethods; Colors and patterns to be painted, anodized, deposited, plated,stamped, printed, etched, embossed, or otherwise used to change thevisual appearance of the eyewear; and in general, quantitativeinformation specified from the user's order automatically converted intofiles or instructions for manufacturing equipment.

FIG. 15 shows an example of a 3D eyewear design 1501 that isautomatically converted into flat patterns of the front 1502, lefttemple 1503, and right temple 1504 to prepare for laser cutting ormachining out of sheet metal or plastic. These parts, along with otherparts from other orders, are automatically arranged to optimizemanufacturing metrics such as the minimizing of material usage orprocess time. The flat patterns also contain geometric informationregarding bend locations 1505 to be used by manufacturing equipment tobend or form the pre-cut parts. The pattern is stored as a digital fileor other media needed to provide the manufacturing equipment withdimensions and instructions. Subsequent operations may include bending,folding, or other forming operations performed on automated equipment.The manufacturing system may use the serialized identifier to determinewhat operation to perform on the part or to obtain the specificationsfor the part at each step. Bend patterns or other computer-readableinstructions are provided to the equipment.

FIG. 16 shows an example of a 3D parametric eyewear model 1601 that wascustomized for a user and the resulting manufactured part 1602 that wasproduced. Parts such as these are created using any of the previouslymentioned manufacturing technologies or other methods known to thosefamiliar in the art.

As to manufacturing, FIG. 1B, step 117, describes the computer systemcontrolling manufacturing equipment and personnel. The computer systemmay sequence a plurality of manufacturing equipment, aided or unaided byhumans. As an illustrative example, the computer system may provide aset of instructions to perform the following sequence to make a metaleyewear frame:

Instructions for robot to pull required material and supply it to alaser-cutting machine or a CNC machine. In parallel, instructions sentto lens manufacturing equipment to surface, polish, mark, coat, and edgelenses. Instructions and tool path for laser cutting machine to cutshape of eyewear and mark with logo or other decorative marking.Instructions for robot to transfer lasercut part to bending and stampingmachine. Instructions for bending and stamping machine to shape eyewearto desired final shape. Instructions for robot to transfer part topolishing machine. Instructions for polishing machine to finish part.Instructions for painting, coating, anodizing, printing, or coloring theeyewear. Instructions for robot to sort finished parts and associateeyewear and lenses. Instructions for human operator to assemble eyewearand lenses, nose and ear pads, and perform final inspection.Instructions for robot to package and label finished product forshipping.

The previously mentioned instructions are one sequence for one customproduct. To enable successful manufacturing of multiple custom products,the computer system controlling the manufacturing process creates asequence of commands for each stage of the process for each custom partbeing produced. FIG. 32 illustrates a block diagram showing a processflow for custom one-up products. Starting at 3201, orders are receivedover time for custom eyewear 1 at 3202, custom eyewear 2 and 3203, andcustom eyewear 3 at 3204. After the orders are received, each eyewearreceives a serial number at 3205. The computer system groups the partsinto batches 3206 and 3207 for laser cutting based on machineavailability, open orders, shifts, and other data. The computer systemprovides instructions to the laser cutter for each batch to cut theparts. So while a custom product moves from the laser cutter to the nextstep, the laser cutter receives instructions for the next batch ofcustom products. After laser cutting, the computer system provides asequence of instructions for each part, one after the other, to abending machine 3208. As each part finishes on the bending machine, thecomputer system provides instructions to a stamping machine 3209.

In one embodiment, the computer system generates instructions forquality control or inspection. The computer system creates templates forhuman inspectors to use, dimensions or pass/fail criteria forinspections. Since each part is unique and one-up, creating uniqueinspection criteria is important. The computer system may also provideinstructions to automated inspection consisting of the dimensions,properties, or criteria for each individual product. Additionally, thecomputer system may provide data or a model of the user's anatomy tomanufacturing equipment to produce an inspection or assembly fixture.For example, a 3D printed model of the user's ears and nose may begenerated to ensure the final product model fits appropriately with theuser.

Subcontractors or multiple manufacturing sites may be used in any of thepreceding steps, and the computer system in one embodiment automaticallyhandles the preparation of order information and/or manufacturinginstructions/schematics. Finally, in step 118 of FIG. 1, the customeyewear is shipped to the user.

Alternate Shopping Systems

The following embodiments describe alternate or additional systems andmethods to supplement or enhance the previous description.

In-Store System

The method and system described to create custom products and eyewear isuseful to have within a retail store, optometrist office, or otherphysical location. The system and method in part or in whole iscontrolled by an a customer, optician, optometrist, sales-person, orother professional assisting a user with the selection and purchase ofthe best frame and lenses in an office or retail location or throughremote assistance through a computer network. FIG. 26 illustrates anexemplary method of shopping for custom eyewear with a system in astore. FIG. 27 illustrates an exemplary computer system. The in-storecomputer system 2730 is used by customer 2700 with optional assistanceby an in-store or remote professional 2710. The computer system 2730 isconfigured with an image capture device 2720 and display 2740. Thecomputer system optionally has calibrated imaging devices to measurecolor for custom color matching an object of the user's for the customeyewear material. The in-store computer system is configured with a datatransfer connection 2750 to the manufacturer's system 2780 and 2790 andoptionally to the computer's computer system 2770 and the professional'sstore computer system 2760, which may contain the user's information,info, prescription, etc.

If the process was started at the professional's store or office, theuser's personal computer system has access to the user's image data andeyewear inventory after a session with a professional, so the user couldaccess this information at a later time. For example, they couldcontinue shopping at home after getting the initial model andcustomization setup completed at the store. The computer system may alsobe configured to work with optometry devices to measure prescriptioninformation and automatically incorporate the measurements into thecomputer system such that no manual entering of prescription data isneeded. A further advantage of an in-store system is the ability tocreate a more controlled and higher-quality image capture and displaysystem. With a kiosk or computer system designed specifically for thepurpose of capturing image data and displaying custom previews, moreadvanced for specialized hardware components could be used, such asmulti-camera systems or depth sensing cameras with calibration.

FIG. 27 illustrates an exemplary method. In this embodiment at 2701, acomputer system configured with a camera or imaging device is used toacquire image data of a user. The computer system may optionally befurther configured with reference targets, multiple or calibratedimaging devices, depth devices, wearable reference targets such aseyewear, or calibrated distances and positioning devices to ensure thescale of the user is measureable by the computer system. At 2702, thestore or office professional may assist the customer with using thecomputer system and acquiring image data. At 2703, the computer systemreconstructs an anatomic model of the user's face based on the imagedata. At 2704 and 2705, the computer system optionally has an inputdevice that enables a store professional, doctor, or other person toinput additional anatomic data, such as physical measurements,prescription information, etc. The computer system computer systemautomatically configures or adjusts custom eyewear models the user forsize and fit 2707 and style 2708. At 2709, the computer system createscustom products and co-registers the anatomic model with the originaluser images such that the model coordinates and camera position alignthe face model with the pose, position, and scale of the images of theuser's face. At 2710, the computer system aligns an eyewear model withthe user model and images and render a preview of eyewear models on theimages of the user. At 2711, the computer system optionally has orconnects to a rapid prototyping system (3D printer, CNC cutter, etc) tocreate a physical prototype or preview for the user. At 2712 and 2713,the computer system has input devices that enable the user or storeprofessional to adjust, update, or configure the custom eyewear models.the computer system has an input device to enable a user or storeprofessional to select and try various eyewear models. At 2714, thecomputer system, and optionally the professional, may recommend if theeyewear is not well suited to the customer. At 2715, the computer systemcalculates data about price and manufacturing time. At 2717, the user orstore professional to select and try various eyewear models. At 2716,the customer may select to order the custom eyewear. At 2718, thecomputer system transfers the final eyewear model and user informationto a manufacturer's computer system via a network connection or otherform of electronic communication such that the manufacturer can producethe custom eyewear. At 2719 and 2720, the manufacturer's computer systemand manufacturing system preprocess the eyewear model and informationand produce custom eyewear. At 2721, the custom eyewear is completed andshipped to the customer or is ready at the store location for pick-up.

Sharing Data and Design Access

In another embodiment, the user provides access to his image data andanatomic model to another party, such as a friend, family member, eyecare professional, or fashion consultant. The user enables the computersystem to transfer their image data, and optionally other informationsuch as preferences, eyewear models, and settings over a network or datatransfer technology to another computer system. This transfer is donewith a hyperlink, authenticated login, or other mechanisms that are sentdirectly to another person through one of a variety of communicationforms, such as email, digital messages, social networking, cloudstorage, etc. The other party then adjusts, customizes, and previewseyewear on the original user's face model or image data. The other partythen saves favorites and eyewear designs and then sends back images,designs, views, customizations, suggestions, notifications, etc to theoriginal user. The original user then uses his computer system topreview the eyewear designed and fitted for him by the other party. Thisembodiment has a huge advantage of allowing the user to crowdsource thedesign of their eyewear to other people, potentially magnifying thediversity and quality of the designs they receive for previewing. Inthis case, they have both the power of computer-driven algorithms andhuman-driven design.

In an exemplary embodiment, the user sends a plurality of image data orinteractive models of himself with previews of eyewear. The image dataor models is sent from the user's computer system to another computersystem via a computer network of other information transmission systemthrough one of a variety of communication forms, such as email, digitalmessages, social networking, cloud storage, etc. The computer systemthen allows an authorized person or people to provide responses,ratings, messages, and other forms of feedback to the original user.

In another embodiment, the system is used by eyewear designers orfashion brands to create their own lines of eyewear. A large start-upcost exists for building a new line of eyewear since parts must beordered in bulk from traditional manufacturing methods, high-fidelityprototypes are expensive, and many combinations of styles, sizes, andcolors must be ordered and held in inventory before any sales are made.The system described herein could be used by a designer to create a setof designs with varying colors, shapes, sizes, and other features. Adatabase full of user image data, anatomic models, and preferencesprovides an extraordinary means to test and preview eyewear across alarge sample of people. Samples of the designs may be provided and asusers view and want to order the designs, an on-demand manufacturing anddelivery method could be used so the designer or fashion brand wouldnever need to carry inventory.

In another embodiment, the system may be used without the image analysisportion if an eyecare professional takes physical measurements and usesthe computer system and to enter anatomic data about the user into asystem that generates custom designs with configurable eyewear models.The professional or user may then provide preferences and refinementsand have the eyewear manufactured as previously described.

Additional Products

In another embodiment, all the methods and techniques described hereinare applied to the customization, rendering, display, and manufacture ofcustom eyewear cases. A user could select from a plurality of materials,colors, designs, shapes, and features and see an accurate rendering ofthe case on his display. Moreover, the case can automatically be sizedto fit the custom eyewear designed such that there is not an excess offree space within the case that would allow the eyewear to bouncearound—the case can be automatically designed to custom fit the eyewearsuch that it minimizes the size of the case and increases the case'sability to protect the eyewear in transport. The case color, style, andmaterials, and method of manufacture can also be matched to those usedto make the custom eyewear. Custom text, such as the name of the user,is engraved or marked on or in the case. The same eyewear manufacturingtechniques described herein can also be used to manufacture the customcases.

Those skilled in the art will recognize that the systems and methodsdescribed herein may also be used in the customization, rendering,display, and manufacture of other custom products. Since the technologydescribed applies to the use of custom image data, anatomic models, andproduct models that are built for customization, a multitude of otherproducts is designed in a similar way, for example: Custom Jewelry (e.g.bracelets, necklaces, earrings, rings, nose-rings, nose studs, tonguerings/studs, etc), Custom Watches (watch faces, bands, etc), CustomCufflinks, Custom Bow Ties and Regular Ties, Custom Tie Clips, CustomHats, Custom Bras, Inserts (pads), and other undergarments, CustomSwimsuits, Custom Clothing (jackets, pants, shirts, dresses, etc),Custom Baby Bottle Tips and Pacifiers (based on scan and reproduction ofmother's anatomy), Custom Prosthetics, Custom Helmets (motorcycle,bicycle, ski, snowboard, racing, F1, etc), Custom Earplugs (active orpassive hearing protection), Custom Audio Earphone (Headphone) Tips(over-the-ear and in-ear), Custom Bluetooth Headsets Tips (over-the-earor in-ear), Custom Safety Goggles or Masks, and Custom Head-MountedDisplays

As an example embodiment of another product, the following system andmethod describe a custom helmet product. Refer to FIG. 33.

In accordance with an embodiment, methods are disclosed for creatingcustom helmets. One method includes acquiring, using at least onecomputer system, image data of a user (two users with different headshapes are shown at 3301 and 3302); determining, using at least onecomputer system, anatomic details and/or dimensions of the user;configuring (eg, custom shape, size, dimensions, colors, finish, etc),using at least one computer system and anatomic data of the user, a newcustom helmet model for the user (a configurable helmet model 3303 isshown with protective element 3304 and strap 3305); applying, using atleast one computer system, a configurable helmet model to the image dataor anatomic model of the user; previewing, using at least one computersystem, images of the user with the configurable helmet model (customhelmet models 3306 are shown on the users, adapted to their unique headshapes); optionally adjusting and updating the preview, using at leastone computer system and/or user input, the configurable helmet modelproperties (eg, custom shape, size, dimensions, colors, finish, etc);preparing, using at least a computer system that executes instructionsfor manufacturing the custom helmet based on the previewed model; andmanufacturing, using at least one computer system and manufacturingsystem, the new custom helmet.

In accordance with an embodiment, systems are disclosed for creating acustom helmet. One system includes an image acquisition deviceconfigured to obtain image data of a user; an input device configured toreceive instructions from a user; a display configured to display imagedata to a user; a manufacturing system configured to produce a customhelmet; a digital storage device to store instructions for creating andpreviewing custom helmet; a processor configured to execute theinstructions to perform the method including: acquiring, using at leastone computer system, image data of a user; determining, using at leastone computer system, anatomic details and/or dimensions of the user;configuring (eg, custom shape, size, dimensions, colors, finish, etc),using at least one computer system and anatomic data of the user, a newhelmet model for the user; applying, using at least one computer system,a configurable helmet model to the image data or anatomic model of theuser; previewing, using at least one computer system, images of the userwith the configurable helmet model; optionally adjusting and updatingthe preview, using at least one computer system and/or user input, theconfigurable helmet model properties (eg, custom shape, size,dimensions, colors, finish, etc); preparing, using at least computersystem, instructions for manufacturing the custom helmet based on thepreviewed model; and manufacturing, using at least one computer systemand manufacturing system, the new custom helmet.

1-28. (canceled)
 29. A computer-implemented method for customizing aparametric model of an eyewear product, the method comprising:capturing, via one or more sensors, a plurality of images of a user;processing the plurality of captured images to extract anatomical dataof the user; generating an anatomic model of at least a portion of theuser and the parametric model of the eyewear product based, at least inpart, on the extracted anatomical data; generating a presentation of theparametric model superimposed on the anatomic model of the user;receiving one or more inputs from the user to adjust at least oneconfigurable portion of the parametric model; and updating theparametric model based on the received user input.
 30. Thecomputer-implemented method of claim 29, wherein generating theparametric model of the eyewear product, further comprising: trackingone or more actions of the user while selecting, customizing, andpreviewing the parametric model; processing historical data associatedwith the user to train a machine learning model to predict preferenceinformation for the user; and generating the parametric model in ashape, size, style, material, and color based, at least in part, on thepredicted preference information of the user.
 31. Thecomputer-implemented method of claim 29, further comprising: tracking,in real-time, facial movements of the user; and generating, inreal-time, a presentation of the updated parametric model superimposedon the anatomic model in a user interface of a device, wherein positionand/or orientation of the updated parametric model and the anatomicmodel is adjusted, in real-time, to represent the facial movements. 32.The computer-implemented method of claim 29, wherein the at least oneconfigurable portion of the parametric model corresponds to amanually-adjustable portions of the eyewear product.
 33. Thecomputer-implemented method of claim 32, further comprising: processingthe one or more inputs from the user to adjust the at least oneconfigurable portion of the parametric model to determine the adjustmentis beyond a pre-defined physical constraints of the manually-adjustableportions of the eyewear product; and generating a notification in a userinterface of a device to alert the user that the adjustment results inan improper fit.
 34. The computer-implemented method of claim 29,wherein processing the plurality of captured images, further comprising:analyzing the plurality of captured images to iteratively perform asequence of feature detection, pose estimation, alignment, and modelparameter adjustment; and applying a face detection and pose estimationalgorithm to determine a position and direction of a user's face. 35.The computer-implemented method of claim 30, wherein the anatomic modelis distorted to fit a user's face, further comprising: parameterizingthe anatomic model and representing the anatomic model as a mesh,wherein a resolution of the mesh is altered based on curvature,location, and features on the user's face.
 36. The computer-implementedmethod of claim 35, further comprising: analyzing, via the machinelearning model, the plurality of captured images to detect facialfeatures; and aligning and adjusting the anatomic model with thedetected facial features to minimize error between the detected facialfeatures and the mesh.
 37. The computer-implemented method of claim 29,further comprising: generating a presentation of visual markers toindicate the at least one configurable portion of the parametric model;and generating a presentation of targets, axes, alignment markers, orrulers to aid the user in providing the one or more inputs.
 38. Thecomputer-implemented method of claim 30, further comprising: storing theplurality of captured images, the historical data, or a combinationthereof in a database, wherein the plurality of captured images arelabeled with location data, direction data, orientation data, datarelating to specific facial features, or a combination thereof; andprocessing the stored plurality of captured images, the historical data,or a combination thereof to build a training database for prediction andcustomization of the eyewear product for returning users or new users.39. A system for customizing a parametric model of an eyewear product,the system comprising: a data storage device storing instructions forcreating the customized eyewear product; and a processor configured toexecute the instructions to perform a method comprising: capturing, viaone or more sensors, a plurality of images of a user; processing theplurality of captured images to extract anatomical data of the user;generating an anatomic model of at least a portion of the user and theparametric model of the eyewear product based, at least in part, on theextracted anatomical data; generating a presentation of the parametricmodel superimposed on the anatomic model of the user; receiving one ormore inputs from the user to adjust at least one configurable portion ofthe parametric model; and updating the parametric model based on thereceived user input.
 40. The system of claim 39, wherein generating theparametric model of the eyewear product, wherein the processor isfurther configured for: tracking one or more actions of the user whileselecting, customizing, and previewing the parametric model; processinghistorical data associated with the user to train a machine learningmodel to predict preference information for the user; and generating theparametric model in a shape, size, style, material, and color based, atleast in part, on the predicted preference information of the user. 41.The system of claim 39, wherein the processor is further configured for:tracking, in real-time, facial movements of the user; and generating, inreal-time, a presentation of the updated parametric model superimposedon the anatomic model in a user interface of a device, wherein positionand/or orientation of the updated parametric model and the anatomicmodel is adjusted, in real-time, to represent the facial movements. 42.The system of claim 39, wherein the at least one configurable portion ofthe parametric model corresponds to a manually-adjustable portions ofthe eyewear product.
 43. The system of claim 42, wherein the processoris further configured for: processing the one or more inputs from theuser to adjust the at least one configurable portion of the parametricmodel to determine the adjustment is beyond a pre-defined physicalconstraints of the manually-adjustable portions of the eyewear product;and generating a notification in a user interface of a device to alertthe user that the adjustment results in an improper fit.
 44. The systemof claim 39, wherein processing the plurality of captured images,wherein the processor is further configured for: analyzing the pluralityof captured images to iteratively perform a sequence of featuredetection, pose estimation, alignment, and model parameter adjustment;and applying a face detection and pose estimation algorithm to determinea position and direction of a user's face.
 45. The system of claim 40,wherein the anatomic model is distorted to fit a user's face, whereinthe processor is further configured for: parameterizing the anatomicmodel and representing the anatomic model as a mesh, wherein aresolution of the mesh is altered based on curvature, location, andfeatures on the user's face.
 46. A non-transitory computer readablemedium for use on a computer system containing computer-executableprogramming instructions for customizing a parametric model of aneyewear product, the method comprising: capturing, via one or moresensors, a plurality of images of a user; processing the plurality ofcaptured images to extract anatomical data of the user; generating ananatomic model of at least a portion of the user and the parametricmodel of the eyewear product based, at least in part, on the extractedanatomical data; generating a presentation of the parametric modelsuperimposed on the anatomic model of the user; receiving one or moreinputs from the user to adjust at least one configurable portion of theparametric model; and updating the parametric model based on thereceived user input.
 47. The non-transitory computer readable medium ofclaim 46, wherein generating the parametric model of the eyewearproduct, further comprising: tracking one or more actions of the userwhile selecting, customizing, and previewing the parametric model;processing historical data associated with the user to train a machinelearning model to predict preference information for the user; andgenerating the parametric model in a shape, size, style, material, andcolor based, at least in part, on the predicted preference informationof the user.
 48. The non-transitory computer readable medium of claim46, further comprising: tracking, in real-time, facial movements of theuser; and generating, in real-time, a presentation of the updatedparametric model superimposed on the anatomic model in a user interfaceof a device, wherein position and/or orientation of the updatedparametric model and the anatomic model is adjusted, in real-time, torepresent the facial movements.